You really should do one of these with Stan from Lynx as I think some of the issues raised here have been improved upon as it’s got to the production stage
Great interview. Yes, some of it's like a bucket of cold water but we still need to hear it, as pretty much all of what we hear these days is some kind of marketing plan being worked on us with little to back it up in the real world
The more issues I learn about in both AR and VR the more amazed I am at Tilt5 and their solution which basically sidesteps almost all of them and replaces them with one that you can actually live with (in my opinion).
But it also unfortunately "sidesteps" 99% of the value of the mainstream AR concept. Even the tabletop gaming feature is incomplete compared to the "dream of great AR", because: 1. you need retroreflector on the table (hassle+reduced portability) 2. no princes Leia type of holograms, everything gets cut off at the frame of the reflector. Tilt5 is effectively a 3D TV laying on the table on its back + headtracking. When you think about it this way it becomes less exciting. If your AR glasses can't even put a little gremlin on your desk in front of your furniture then can it even be called "AR"?
I used to hate when people would say "VR isn't there yet." year after year, expecting that VR should be some full dive experience out of the gate or it's useless and ignoring the fact that it will NEVER get to that point if it's not adopted eventually before. With AR though, I feel it should be it's own separate entity for a while, because it actually isn't there yet. I think they're splitting the focus too much on integrating AR into VR when AR is still way too young. My own opinion is that Cambria isn't interesting to me because most of their focus seems to be AR, and honestly the thought of having a big headset on and using it for AR seems incredibly unappealing to me, a novelty at best that I would never use. It could wind up hurting both AR and VR as a result.
the incorporation of AR into VR is kind of inevitable. they already have some passtrough apps for quest 2 so they'll only improve passthrough on their next headsets and make more software for it. I think the separation should be more like optical vs passthrough AR like they pretty much do in this video. only "risk" is that passthrough AR will reach an acceptable level while optical AR is trying to play catch up but you can't really prevent that nor should you. optical AR devs just need to get their asses in gear XD
I love his comments on the sort of collective insanity that can happen when companies are racing against each other. You see it with many other things, e.g. Facebook is pushing a metaverse and therefore everyone else needs a metaverse, even though none of them know what it is.
That said, it's not like I think passthrough AR won't have any use cases. It's clearly useful for e.g. having your hands, keyboard, HOTAS, etc etc overlaid on the VR experience, or say quickly switching (fully or partially) to passthrough to grab your drink/coffee or attend to any other "real life" thing. But I do wonder how many non-gimmicky consumer applications there are for "pure" passthrough AR experiences and how much people will be willing to tolerate the "twilight zone"-like single focus reality (as I've heard it described) that passthrough AR creates. It seems odd to me to push this as its own class of devices rather than just as a complement to VR.
Physicist here. I agree to Karl. That doesn't mean we have to drop AR out. Consumer market should actually focus on pure VR techs and apps. AR tech is too premature for consumer market, it's actually well suited for research, experimentation and prototyping. And, please, don't label Tilt5 as AR tech. :)
@@Voidapparate No dynamic matching between reality and CGI perceived as a whole = no AR. It's just 3D projection on special horizontal screen. No projection on special screen, no Tilt5. You're not augmenting reality. Otherwise even 3D cinema would be AR. Or whatsoever projection would be AR. Or whatsoever drawing or artifact made by humans would be AR. Period. You disagree? You're welcome, but don't try to convince me. No discussion please.
@@videogamesart8057 "No dynamic matching between reality and CGI perceived as a whole = no AR. Its just 3D projection on special horizontal screen. No projection on screen, no Tilt5. You're not augmenting reality. Otherwise even 3D cinema would be AR. Or whatsoever projection would be AR. Or whatsoeverdrawing or artifact made by humans would be AR. Period. You disgree? You're welcome, but don't try to convince me. No discussion please." - 🤓 Btw I'm just memeing if you haven't noticed yet.
He sounds like a true expert, it would be great to bring him back and get his opinion on the apple vision pro. Its also kinda sad he said it is physicially impossible to get good optical AR
So will the Lynx have actually a lower effective resolution than the already low resolution of the panel? It looks like 4x lower if it's split that way which would be kinda insane.
26:06 I think that's why I love VR more than AR: VR takes you to another world whereas AR just changes your world. Unfortunately, there are people who are constrained by their environment to fully take advantage of AR.
He seems pretty skeptical about XR passthrough but am 100% sure it will be the way of the future. Its already somewhat good in Quest2. You can only imagine how good it will get in future headsets.
He does make a few of good arguments, 1 of them breaking a few laws of physics :P. Warping with software could counter this in a few years, but not good enough to make it look like real life (and thus a strain on eyes/brain if used for more hours). What do you think the solution is?
@@PandaSingstheGames I feel like you get these older guys who say they been working on VR and AR for the last 20 to 30 years. Ok, that may be true... but you haven't been around to see what happens when billions of dollars get pored into research and development. Old ways that did not work then does not mean they won't work in the future when proper technologies and investment are in place.
@@Maustah I have spoken to my functional Neurologist about VR use and we had an interesting conversation, one way to long and in depth for a YT comment. People should make it a practice for meditation after prolonged VR use, unless you have O type blood and then you're just fucked. He recommended if you have this type blood you shouldn't do VR at all because it can lead you towards dementia. Also people have a tendency to stop breathing for moments inside VR, so don't forget to breath people.
Thank you for this content! Way better than Facebook's Zuck telling me he has cool stuff in his basement, but i won't get to ever try it cause i'm not his Facebook friend.
I'm a big fan of pass-through AR because the virtual objects just feel so much more real as they do a better job merging the virtual and physical content and can display perfect blacks. The ZED Mini was just super great with an Oculus CV1, it had bear perfect realtime pixel-level occlusion at 60fps from day 1, and has just been getting better over the years as they've improved their algorithms. I just wish they would release a new version that could do higher resolution than 720p.
this was a great guest. i'm curious if he sees any more promise in using brain chips to do AR/VR in the next couple decades, than he does in optics. Neuralink goes into first human trials this year (if it stays on schedule). and of course there are a lot of steps between that and mass consumer approval, but given that the FDA has given it breakthrough status it's not inconceivable to see that happening in the 2030's or 2040's latest. There's the additional issue of how much brain coverage/penetration you need to properly fool all the senses. This could be an absurdly complicated problem, but it also could be surprisingly easy. If it's the latter, brainchip AR/VR becomes basically a software issue rather than hardware. Collect all the data on how every visual happens in the brain, then reproduce those same signals in the order you need for your game/app and send it to the brain. I know that sounds like a lot, but based on his skepticism and level of physical problems with creating complex hardware that perfectly fits a human form factor, not to mention all of the other problems outside of visual (like vestibular for one): it doesn't seem far fetched to me that this problem could be quicker solved with data/brain science than optical/electrical engineering
I'm not very smart in this but what if we could bend the light kinda like a mirror does? somehow a device would take available light either created by itself or just in the enviroment, bend and make reflections and maybe use kinda what happens when light passes through water to make rainbow, thus rgb
I think screen resolution is the "low hanging fruit"; it's the easiest thing to seek a solution for and we're getting to "eye level" detail relatively fast (I see at least one set being in consumer hands by the end of the year.) The challenge is light with debt perception- that is, seeing light in and out of focus. I would believe the issue with camera and eye placement should be able to be fixed in software- assuming the hardware can center it's optics properly to the users eyes.
Hmm, I really doubt we'll have human-eye resolution by the end of the year. But maybe within 2 or 3 years. Cambria is the only seriously capable consumer headset due to release by the end of the year, and it will probably have about 30ppd, compared to the 60ppd or so of the human eye
Yup, I honestly don’t see any reason for full consumer AR aside from niche experiences. At most maybe a pair of glasses that let you see stuff on your phone. Like a HUD. Letting you know time, allows you to see emails and text without having to remove your phone. But full ass AR where you augment the world around you? Not really. Not any time soon anyways. Nobody is going to be driving a car or taking a stroll with a huge VR goggles just to change how the world looks? I think pass through is a neat idea for VR users for safety and to be able to recognize objects in the real world without having to take the helmet off. Just don’t see any killer apps that will want you to AR your world that you couldn’t do in VR.
As a software developer I'm waiting for an AR device thats light and comfortable to wear all day, with high enough resolution to read text on virtual displays with a fov wide enough to not clip virtual objects to noticeably (90 degrees plus). It would also need bluetooth for input devices and maybe hand tracking would be nice for virtual 'touch' displays and inside out tracked. I wont really care to much about accurate color representation, crazy high refresh rates or even brightness to a degree. I wouldnt even mind if it was a tethered device, probably have to be for all day power. The Nreal seems to come close but its FOV is to small for the asking price, plus its damb hard to actually buy here in spain. only vodafone retails it and then only with a contract with them and i certainly dont want the contract with vodafone. I'd want a full VR headset for games, no interest in optical AR for gaming. Color, refresh, fov, 2-4 hours battery life non tethered with high accuracy outside in tracking for gaming.
well nreal fov is not too low compared to the price...... it's the same fov as headset worth thousands of dollars ....and the image quality is better then those what about something like cambria with 40ppd and passthrough ...it will have issues ...but it will be pretty great
@@pumpuppthevolume i guess its down to personal opinion. 52 degrees is to low for me to use comfortably. I'm not saying there's any other headset that is at a usable (for me) fov, just this ones to low for me personally. I'm just waiting for something to become available that's something more like 90 degree's. I have exceptional peripheral vision so even the VR headsets with 100+ degree's consciously cuts off my vision very notably, and if I'm paying a large sum of money for a tool that i'll use for multiple hours a day. I want it to be comfortable and not cut things off mid vision or narrow my vision.
@@thatgamingbrit yes it's not big fov ....I'm saying it's cheap and it's as good as hololens 2 and magic leap 1 and they cost multiple times that amount......which is quite the achievement and it's much smaller and lighter than those professional headsets
I don't get that hyperloop is physically impossible analogy. Hyperloop is almost like a metaverse type of buzzword and can mean many different things (there are no set in stone specs and methods, just the general idea of high speed tunnels). It is 100% possible. Even proto-hyperloop "trains" were tried something like 100 years ago. Let's not forget that 10 years ago when SpaceX consulted experts and scientists outside of company on retropropulsive landing for orbital boosters they got literally 100% of negative replies. Not even a single one believed it could work. They tried anyway. It worked. They just relaunched a used booster for 100th time a few days ago and broke record of 3 landings in 36 hours...
I've seen a lot of hype for AR, but I've never gotten a good vision of WHAT people REALLY want to do with AR. I can see it working on a Car's windshield... I can see it working for a ROBOT that needs to see to get around.. but an AR headset with pass-through seems JUST okay. The best USE CASE for AR that I've seen is Tilt 5, and that's because it's got a very obvious use case... play games around a table... it can do that with a light headset and it works now. What EXACTLY do you want to do with a Cambria or advanced AR headset beyond that? To me, VR is much more compelling, especially because a DARK room is better for watching TV/media, so having a screen appear in my bright room isn't ACTUALLY an improvement. I DO think that content created for VR could be a huge game changer -- especially camera-arrays that can video-record 3D spaces during family events. If I can set up 3 cameras on tripods and have a perfect VR representation of my children's first christmas... I want to do that. Then later I can walk around the scene, look at the tree, watch the kids open their presents from any angle... That seems so DOABLE, and a major improvement over vertically-filmed cellphone cameras.
the concept of using AR extends far beyond entertainment, in a sci-fi futuristic world your AR glasses indistinguishable from normal ones will highlight the areas where caps/public transports will arrive, give you information regarding your most optimal path, show you the current state of a traffic light in the distance, guide you through a market to the product you're looking for, highlight your friend's location in the crowd during a concert, remind you to take your medicine, give you the classic HUD map experience instead of having to look constantly at your phone. In short, it's going to completely make phones redundant and surpass them in every level, of course we're not just lacking in optical department to make this viable but also on other hardware/software sides.
couldn't they make like a very thin sandwich of a pancake lens, a screen and a camera and then put 2 of those in a headset? so if you move the lenses/screens to adjust for your IPD the camera's also perfectly align with the center of your eyes? like they'll be a few mm/cm's in front but everything will be way better aligned in a pretty simple way....
8:55 The amazing Karl predicting or quietly inferring that Bradley will probably meet the afterlife early in some kind of out-of-hand tumultuous waifu-related/"stupid gaijin" type accident.
If you want to see one of the latest innovations in Passthrough AR, check out Passthrough Windows: passthroughstudio.gumroad.com/l/passthroughwindows. IU
@5:27 I don't know if this guy has not actually tried Passthrough AR (maybe he only tried the Quest 2 or Index passthrough *oof*) but that is just not true. The depth isn't "real" but it certainly appears that way to the user, just like with VR.
Karl is a bag of hammers, great job on the interview though Brad. I do like the interviews, maybe next time we find someone who can find something positive to say about the industry we all love. Karl Guttag is a walking talking Black Pill.
I have someone already lined up! Next interviewee will be probably the complete opposite of Karl's tone. And has some cool things to say about BCI hardware like Galea
wow people really are allergic to critical thinking on youtube. He didn't criticize "the industry" he criticized AR. Nothing he said means there can't be great VR with AR elements, its just that AR tech wont be ready to carry a product *on its own* for a very long time, which was already obvious years ago. Even if everything he said was wrong there are no batteries small enough to fit on glasses that could last longer than a few minutes anyway. AR in itself was NEVER a realistic expectation for this decade, people were just really desperate to justify and rationalize Apple entering the market. VR is a massive challenge right now, but also already a big success.
You do understand he's an enthusiast who states on his blog that he *is* excited about the future? I want to be surprised that someone would be incapable of taking discussion of the downsides of something without seeing them as opposition, but it's pretty on brand for someone who'll use Black PIll unironically.
Since he mostly does optical AR has he actually tried much consumer AR stuff? He keeps talking about issues with it but alot of the issues arent that big of a deal. Even the quest 2 passthrough with it stitching multiple images together and being black and white is still pretty decent and easy to get used to IMO. If quest passthrough was color and a bit higher resolulution i feel like i could do everything i need to im my daily life in passthrough.
The issue is there really isn’t any consumer AR to try yet. Only things that startups show off. But I have a feeling he might not have tried the quest 2 passthrough
quest 2 passthrough is terrible, grayscale and very blurry, hard to use for more than a few minutes if one has to actually look at the real world for too long. If any AR device tried to enter the market with Quest 2's passthrough it would be crushed.
@@Wobbothe3rd yea I know it's blurry but I was addressing the alignment issue that was mentioned in the video. I've never experienced any of the uncomfortable side effects he mentioned from passthrough. The blurryness obviously becomes uncomfortable when you're trying to read something small like your phone but if the camera resolution were increased, I could see it being fairly comfortable to use
Technically all AR, even phone-based AR, already can be useful for some things. Hololens is already used for work. The problem is passing that threshold of positive value for many millions of consumers instead of being a niche device. The founder of Oculus has a blog post claiming that current VR is still not good enough for many people even if it was completely free, especially for long time use. Big corporations like Meta and Apple aren't chasing useful AR, they are chasing better-than-smartphone AR for billion+ people. And that's the point of this video - it won't be possible even on a decade. On the other had some people were happy with HMDs that we had in the 90s. It's a spectrum.
Great interview. Learned a lot. One thing I think is an unfair criticism of passthrough VR is that you put the image all on one focal plane. I don't think that's going to be true for long with varifocal displays like the half-dome prototypes shown by Meta recently.
I think this guy doesn't really understand a lot of the use cases people will use AR for. cuz like he wants to make this perfect AR that mixes the augmented and real world perfectly. while I think a lot of people just want to do VR stuff but they also want to be able to see the world around them a bit. like productivity stuff where you would just want a lot of large screens or other assets floating around you while also seeing a real mouse and keyboard to control them with. if you use that kind of stuff for your job you don't want to be stuck in VR all day while you're sitting in your office. it would be nice if you could look past your screen to see the door and shit. or if you're using it to model stuff in 3d or something, again you don't really need a full VR world surrounding you and being able to see the office (and possibly co-workers working in the same space as you) would be very nice. even if they don't look like picture perfect representations of the real world....
I think the future of AR isn't gaming but instead displays for phones and media. I can't tell you how many times a day I see someone awkwardly supporting their arm to watch a clip or movies on their phones. If you can offer people a nice screen behind a set of nice looking sunglasses without a wire, you have a product. Its coming.. Its almost here with things like the Nreal Air. But its a few years away.
It's about time some of these "AR >>> VR, fancy glasses or bust, nobody wants VR!!!" companies are getting some cold water thrown at them. Sorry dawgs, you'll need to grit your teeth and focus on passthrough like everyone else. No shortcuts!
@@SadlyItsBradley I think the use cases presented in the AR hype are, in reality, virtually non-existent. I was a VR early adopter with a Z-800, later the DK1 and so on. With well over a dozen HMDs so far (including a HoloLens 1) I think it's fair to say that I'm a fan of it. Some seem to think that AR is the next evolution of VR, but I don't see it that way. I know there are some very cool things you can do with AR, but I really have very little interest in it when compared to what can be done in VR. It reminds me a little of the marketing hype around 3DTV. The marketing wanks hyped it as "the new way you'll watch TV" when they should have been selling it as an _additional capability_ (and an extremely cool one IMO) that you can use now and then for specific things. I see AR as a nifty additional capability for HMDs, but I would have zero interest in an AR only device.
Karl is going to get pretty surprised. Never underestimate what can be done now that we have insane neural nets and compute to boot. The same thing was pretty much said for inside out tracking. It will never work, no one will code a decent enough solution. Half write. Humans dont write any of that code anymore, its all handled by neural nets, and no human would of been able to code from scratch the same solutions. Working voice to text ? Never going to happen, the words of the top tier people working in the field 15 years ago. They were half right, once again. No human is going to code an AI that can do extremely perfect audio to text, its done by Neural Nets.
In the end, AR and anywhere-computing will be ubiquitous. I like being immersed in movies and games on a Friday night, but I use my computer (and phone) for obscene amounts of time every day. The "shooty games on the wall" Karl mentioned applies to AR as much as Doom does to modern computing. I wonder though, which side of "the great filter" are the perfect AR glasses on? Seems like we got a pristine world and wrecked it in less than 100 years.
The problem with VR, AR, MR, XR, ZR(?) is the R part. It seems the companies working on these projects do not have... (for the lack of a better description) A classically, hard core, extremely well-trained Physicist in their teams. A person to point out and explain that there is no Real R... with glasses or not... Trying to put together a piece puzzle, with pieces from 3 different boxes... Yeah one will find pieces that seem to have "problems", occlusion/paralax/delays/limitiations to dynamic range/resolution.... And they waste time trying to fit pieces from different boxes. Obviously it is impossible to say what a Physicist would say to help with all this in a youtube video, but here's something more than just the thought experiment above... Go find a colour-blind person and try to talk about their view of reality. AND then try to solve that "problem" :) Wonderful series and interview Brad! Keep up the good work, and waiting on a deep dive of Nreal's Steam thingy :)
sure not everyone will use a ar headset. does everyone has a gaiming pc no. its ok to not apeal toeverybody. imagine the same discusion but with the topic gaming pcs. "its to expensive for kids" "old people will not use it" "the grafic dosnt look 100% fhotorealistic" " oh you can see a pixel on the screen" . it dosnt have to pe 100% perfect. no product is. and not every prodact is for everyone. ther are so many people in the world and we only think about who will not use the device. Gaming was never perfect. In the begining aming was realy nesh. its better to grow naturaly to a bigger audience. many think that it will change the world.but if the expectatioons are so hige. ther is no wonder that somone will be disapointed.
Yes. MR will be just that - geeks, gamers, enthusiast and pro users. But Meta wants billion+ users and replace smartphones. That's a whole different challenge that requires 100x better tech.
@@kazioo2 I think meta is the only one who wants billion users. Gaming pcs / consols are also a big industry. Not every gamer is a neard. But I get whee u coming from.
can't you "just" fix the camera placement thing with reprojection or something? like the quest 2 doesn't have camera's near your eyes at all but the passthrough, while far from perfect, doesn't make me feel weird at all. I can imagine some better version of that in color working decently enough. like ofcourse you won't be fooled into thinking you're looking at the real world but if you can just see the world around you and your keyboard and shit properly it would solve a lot of AR issues already. like this dude seems to be focusing way too much on making the real world look good in AR while IMO you should focus on making the AR content look good cuz that's why you're wearing the thing in the first place. you won't be focusing on the background 99% of the time. so I think passthrough AR is the way forward for the very near future cuz at least there the AR stuff doesn't look like some blurry hard to see ghost image....
@@SadlyItsBradley That explains a lot. VR as it is can provide all kinds of truly amazing immersive experiences and it's only getting better. I have exactly zero interest in AR, however.
VR is sucessful but actually not that big yet. More skepticism and criticism is warranted, even with the Quest 2 selling 10 million plus. VR still has a LONG way go to live up to the hype, and standalone has big tradeoffs compared to PCVR despite the benefits. Karl may turn out to be right in many ways, the jury's still out,so to speak.
AMAZING CONTENT NOBODY ELSE DOES IT LIKE YOU IN THE VR SPACE THANK YOU WEEB OVERLORD
Thank you Brad, it was really nice to learn a bit more about the pros and cons of each method
You really should do one of these with Stan from Lynx as I think some of the issues raised here have been improved upon as it’s got to the production stage
I’d be happy to interview Stan one day 😇
Great interview. Yes, some of it's like a bucket of cold water but we still need to hear it, as pretty much all of what we hear these days is some kind of marketing plan being worked on us with little to back it up in the real world
*Ahhh, been waiting for this one!*
🙂👍🏻
Hope you like it!
The more issues I learn about in both AR and VR the more amazed I am at Tilt5 and their solution which basically sidesteps almost all of them and replaces them with one that you can actually live with (in my opinion).
Tilt 5 is probably the only system that I’ve heard Karl “wow” over. I think that says a lot
But it also unfortunately "sidesteps" 99% of the value of the mainstream AR concept. Even the tabletop gaming feature is incomplete compared to the "dream of great AR", because: 1. you need retroreflector on the table (hassle+reduced portability) 2. no princes Leia type of holograms, everything gets cut off at the frame of the reflector. Tilt5 is effectively a 3D TV laying on the table on its back + headtracking. When you think about it this way it becomes less exciting. If your AR glasses can't even put a little gremlin on your desk in front of your furniture then can it even be called "AR"?
@@kazioo2 Agree. It's not AR. Not 3D tv, it's 3D projection on special horizontal screen
@@kazioo2 Some impressive mental gymnastics required to be this asinine.
I used to hate when people would say "VR isn't there yet." year after year, expecting that VR should be some full dive experience out of the gate or it's useless and ignoring the fact that it will NEVER get to that point if it's not adopted eventually before. With AR though, I feel it should be it's own separate entity for a while, because it actually isn't there yet. I think they're splitting the focus too much on integrating AR into VR when AR is still way too young. My own opinion is that Cambria isn't interesting to me because most of their focus seems to be AR, and honestly the thought of having a big headset on and using it for AR seems incredibly unappealing to me, a novelty at best that I would never use. It could wind up hurting both AR and VR as a result.
the incorporation of AR into VR is kind of inevitable. they already have some passtrough apps for quest 2 so they'll only improve passthrough on their next headsets and make more software for it. I think the separation should be more like optical vs passthrough AR like they pretty much do in this video. only "risk" is that passthrough AR will reach an acceptable level while optical AR is trying to play catch up but you can't really prevent that nor should you. optical AR devs just need to get their asses in gear XD
I love his comments on the sort of collective insanity that can happen when companies are racing against each other.
You see it with many other things, e.g. Facebook is pushing a metaverse and therefore everyone else needs a metaverse, even though none of them know what it is.
That said, it's not like I think passthrough AR won't have any use cases. It's clearly useful for e.g. having your hands, keyboard, HOTAS, etc etc overlaid on the VR experience, or say quickly switching (fully or partially) to passthrough to grab your drink/coffee or attend to any other "real life" thing. But I do wonder how many non-gimmicky consumer applications there are for "pure" passthrough AR experiences and how much people will be willing to tolerate the "twilight zone"-like single focus reality (as I've heard it described) that passthrough AR creates. It seems odd to me to push this as its own class of devices rather than just as a complement to VR.
the hole consept of a metaverse is that all is conected. so if ther are more than one than the concept failed.
Yes!! I've been waiting for this video :D
Sorry it took so long! The Steam VR datamine pushed back these releases a bit haha
@@SadlyItsBradley Ain't no way you said sorry after dropping this banger
Physicist here. I agree to Karl. That doesn't mean we have to drop AR out. Consumer market should actually focus on pure VR techs and apps. AR tech is too premature for consumer market, it's actually well suited for research, experimentation and prototyping. And, please, don't label Tilt5 as AR tech. :)
"Tilt5 is AR and I'm tired of pretending its not" - 🤡
@@Voidapparate No dynamic matching between reality and CGI perceived as a whole = no AR. It's just 3D projection on special horizontal screen. No projection on special screen, no Tilt5. You're not augmenting reality. Otherwise even 3D cinema would be AR. Or whatsoever projection would be AR. Or whatsoever drawing or artifact made by humans would be AR. Period. You disagree? You're welcome, but don't try to convince me. No discussion please.
@@videogamesart8057 "No dynamic matching between reality and CGI perceived as a whole = no AR. Its just 3D projection on special horizontal screen. No projection on screen, no Tilt5. You're not augmenting reality. Otherwise even 3D cinema would be AR. Or whatsoever projection would be AR. Or whatsoeverdrawing or artifact made by humans would be AR. Period. You disgree? You're welcome, but don't try to convince me. No discussion please." - 🤓
Btw I'm just memeing if you haven't noticed yet.
@@videogamesart8057 baaaaait
He sounds like a true expert, it would be great to bring him back and get his opinion on the apple vision pro. Its also kinda sad he said it is physicially impossible to get good optical AR
Gladly seeing another video with mAsteR to discuss AR vs passtruAR.
thank u brad
My pleasure!
So will the Lynx have actually a lower effective resolution than the already low resolution of the panel? It looks like 4x lower if it's split that way which would be kinda insane.
26:06 I think that's why I love VR more than AR: VR takes you to another world whereas AR just changes your world. Unfortunately, there are people who are constrained by their environment to fully take advantage of AR.
He seems pretty skeptical about XR passthrough but am 100% sure it will be the way of the future. Its already somewhat good in Quest2. You can only imagine how good it will get in future headsets.
Lol I agree with you 100%, that dude Karl was just a crotchety ole black pill
Personally, I agree that passthrough AR is the step forward. Especially in getting VR pushed to the heights we VR fiends all want
He does make a few of good arguments, 1 of them breaking a few laws of physics :P. Warping with software could counter this in a few years, but not good enough to make it look like real life (and thus a strain on eyes/brain if used for more hours).
What do you think the solution is?
@@PandaSingstheGames I feel like you get these older guys who say they been working on VR and AR for the last 20 to 30 years. Ok, that may be true... but you haven't been around to see what happens when billions of dollars get pored into research and development. Old ways that did not work then does not mean they won't work in the future when proper technologies and investment are in place.
@@Maustah I have spoken to my functional Neurologist about VR use and we had an interesting conversation, one way to long and in depth for a YT comment. People should make it a practice for meditation after prolonged VR use, unless you have O type blood and then you're just fucked. He recommended if you have this type blood you shouldn't do VR at all because it can lead you towards dementia. Also people have a tendency to stop breathing for moments inside VR, so don't forget to breath people.
What's the smallest available headset than can run Bigscreen Virtual cinema?
High value content right here.
Thank you for this content! Way better than Facebook's Zuck telling me he has cool stuff in his basement, but i won't get to ever try it cause i'm not his Facebook friend.
Glad people enjoy it. Even thought it’s not the most POSITIVE talking points we are mostly used to
I'm a big fan of pass-through AR because the virtual objects just feel so much more real as they do a better job merging the virtual and physical content and can display perfect blacks. The ZED Mini was just super great with an Oculus CV1, it had bear perfect realtime pixel-level occlusion at 60fps from day 1, and has just been getting better over the years as they've improved their algorithms. I just wish they would release a new version that could do higher resolution than 720p.
Can't you correct the misalignment between the camera and the eye in software?
This is what companies do. There is a lot of great research on that too
I have an xr obsession and I'm not sure if your content is helping or hurting
Probably both. Enjoy the drug that slowly kills you
Amazing!
this was a great guest. i'm curious if he sees any more promise in using brain chips to do AR/VR in the next couple decades, than he does in optics.
Neuralink goes into first human trials this year (if it stays on schedule). and of course there are a lot of steps between that and mass consumer approval, but given that the FDA has given it breakthrough status it's not inconceivable to see that happening in the 2030's or 2040's latest.
There's the additional issue of how much brain coverage/penetration you need to properly fool all the senses. This could be an absurdly complicated problem, but it also could be surprisingly easy.
If it's the latter, brainchip AR/VR becomes basically a software issue rather than hardware. Collect all the data on how every visual happens in the brain, then reproduce those same signals in the order you need for your game/app and send it to the brain.
I know that sounds like a lot, but based on his skepticism and level of physical problems with creating complex hardware that perfectly fits a human form factor, not to mention all of the other problems outside of visual (like vestibular for one): it doesn't seem far fetched to me that this problem could be quicker solved with data/brain science than optical/electrical engineering
I'm not very smart in this but what if we could bend the light kinda like a mirror does? somehow a device would take available light either created by itself or just in the enviroment, bend and make reflections and maybe use kinda what happens when light passes through water to make rainbow, thus rgb
I think screen resolution is the "low hanging fruit"; it's the easiest thing to seek a solution for and we're getting to "eye level" detail relatively fast (I see at least one set being in consumer hands by the end of the year.)
The challenge is light with debt perception- that is, seeing light in and out of focus.
I would believe the issue with camera and eye placement should be able to be fixed in software- assuming the hardware can center it's optics properly to the users eyes.
Hmm, I really doubt we'll have human-eye resolution by the end of the year. But maybe within 2 or 3 years.
Cambria is the only seriously capable consumer headset due to release by the end of the year, and it will probably have about 30ppd, compared to the 60ppd or so of the human eye
Yup, I honestly don’t see any reason for full consumer AR aside from niche experiences. At most maybe a pair of glasses that let you see stuff on your phone. Like a HUD. Letting you know time, allows you to see emails and text without having to remove your phone. But full ass AR where you augment the world around you? Not really. Not any time soon anyways. Nobody is going to be driving a car or taking a stroll with a huge VR goggles just to change how the world looks? I think pass through is a neat idea for VR users for safety and to be able to recognize objects in the real world without having to take the helmet off. Just don’t see any killer apps that will want you to AR your world that you couldn’t do in VR.
As a software developer I'm waiting for an AR device thats light and comfortable to wear all day, with high enough resolution to read text on virtual displays with a fov wide enough to not clip virtual objects to noticeably (90 degrees plus). It would also need bluetooth for input devices and maybe hand tracking would be nice for virtual 'touch' displays and inside out tracked. I wont really care to much about accurate color representation, crazy high refresh rates or even brightness to a degree. I wouldnt even mind if it was a tethered device, probably have to be for all day power.
The Nreal seems to come close but its FOV is to small for the asking price, plus its damb hard to actually buy here in spain. only vodafone retails it and then only with a contract with them and i certainly dont want the contract with vodafone.
I'd want a full VR headset for games, no interest in optical AR for gaming. Color, refresh, fov, 2-4 hours battery life non tethered with high accuracy outside in tracking for gaming.
well nreal fov is not too low compared to the price...... it's the same fov as headset worth thousands of dollars ....and the image quality is better then those
what about something like cambria with 40ppd and passthrough ...it will have issues ...but it will be pretty great
@@pumpuppthevolume i guess its down to personal opinion. 52 degrees is to low for me to use comfortably. I'm not saying there's any other headset that is at a usable (for me) fov, just this ones to low for me personally. I'm just waiting for something to become available that's something more like 90 degree's. I have exceptional peripheral vision so even the VR headsets with 100+ degree's consciously cuts off my vision very notably, and if I'm paying a large sum of money for a tool that i'll use for multiple hours a day. I want it to be comfortable and not cut things off mid vision or narrow my vision.
@@thatgamingbrit yes it's not big fov ....I'm saying it's cheap and it's as good as hololens 2 and magic leap 1 and they cost multiple times that amount......which is quite the achievement and it's much smaller and lighter than those professional headsets
@@pumpuppthevolume yep I agree. its a good product for the current market.
How about going the holographic route?
VR to ER... been there, done that 😅
Is he swearing a lot and you silence those out or is his audio cutting out so much?
audio issues..
So sad his audio stream is so choppy....
Nice talk! I hope seeing more AR stuff in the near future. VR starts getting boring somehow.
Very interesting
I don't get that hyperloop is physically impossible analogy. Hyperloop is almost like a metaverse type of buzzword and can mean many different things (there are no set in stone specs and methods, just the general idea of high speed tunnels). It is 100% possible. Even proto-hyperloop "trains" were tried something like 100 years ago. Let's not forget that 10 years ago when SpaceX consulted experts and scientists outside of company on retropropulsive landing for orbital boosters they got literally 100% of negative replies. Not even a single one believed it could work. They tried anyway. It worked. They just relaunched a used booster for 100th time a few days ago and broke record of 3 landings in 36 hours...
I've seen a lot of hype for AR, but I've never gotten a good vision of WHAT people REALLY want to do with AR. I can see it working on a Car's windshield... I can see it working for a ROBOT that needs to see to get around.. but an AR headset with pass-through seems JUST okay. The best USE CASE for AR that I've seen is Tilt 5, and that's because it's got a very obvious use case... play games around a table... it can do that with a light headset and it works now. What EXACTLY do you want to do with a Cambria or advanced AR headset beyond that?
To me, VR is much more compelling, especially because a DARK room is better for watching TV/media, so having a screen appear in my bright room isn't ACTUALLY an improvement. I DO think that content created for VR could be a huge game changer -- especially camera-arrays that can video-record 3D spaces during family events. If I can set up 3 cameras on tripods and have a perfect VR representation of my children's first christmas... I want to do that. Then later I can walk around the scene, look at the tree, watch the kids open their presents from any angle... That seems so DOABLE, and a major improvement over vertically-filmed cellphone cameras.
the concept of using AR extends far beyond entertainment, in a sci-fi futuristic world your AR glasses indistinguishable from normal ones will highlight the areas where caps/public transports will arrive, give you information regarding your most optimal path, show you the current state of a traffic light in the distance, guide you through a market to the product you're looking for, highlight your friend's location in the crowd during a concert, remind you to take your medicine, give you the classic HUD map experience instead of having to look constantly at your phone.
In short, it's going to completely make phones redundant and surpass them in every level, of course we're not just lacking in optical department to make this viable but also on other hardware/software sides.
infinite screens where ever you want
also with passthrough AR you can have as much light or darkness as the screen can produce.
couldn't they make like a very thin sandwich of a pancake lens, a screen and a camera and then put 2 of those in a headset? so if you move the lenses/screens to adjust for your IPD the camera's also perfectly align with the center of your eyes? like they'll be a few mm/cm's in front but everything will be way better aligned in a pretty simple way....
8:55 The amazing Karl predicting or quietly inferring that Bradley will probably meet the afterlife early in some kind of out-of-hand tumultuous waifu-related/"stupid gaijin" type accident.
If you want to see one of the latest innovations in Passthrough AR, check out Passthrough Windows: passthroughstudio.gumroad.com/l/passthroughwindows. IU
Woof
We need content on hololens! Meta has gone beyond pancake.
Plenoptic cameras would help passthrough AR greatly.
@5:27 I don't know if this guy has not actually tried Passthrough AR (maybe he only tried the Quest 2 or Index passthrough *oof*) but that is just not true. The depth isn't "real" but it certainly appears that way to the user, just like with VR.
Karl is a bag of hammers, great job on the interview though Brad. I do like the interviews, maybe next time we find someone who can find something positive to say about the industry we all love. Karl Guttag is a walking talking Black Pill.
I have someone already lined up! Next interviewee will be probably the complete opposite of Karl's tone. And has some cool things to say about BCI hardware like Galea
wow people really are allergic to critical thinking on youtube. He didn't criticize "the industry" he criticized AR. Nothing he said means there can't be great VR with AR elements, its just that AR tech wont be ready to carry a product *on its own* for a very long time, which was already obvious years ago. Even if everything he said was wrong there are no batteries small enough to fit on glasses that could last longer than a few minutes anyway. AR in itself was NEVER a realistic expectation for this decade, people were just really desperate to justify and rationalize Apple entering the market. VR is a massive challenge right now, but also already a big success.
@@Wobbothe3rd Honestly nothing on YT especially the comment section engages critical thinking.
You do understand he's an enthusiast who states on his blog that he *is* excited about the future? I want to be surprised that someone would be incapable of taking discussion of the downsides of something without seeing them as opposition, but it's pretty on brand for someone who'll use Black PIll unironically.
Since he mostly does optical AR has he actually tried much consumer AR stuff? He keeps talking about issues with it but alot of the issues arent that big of a deal. Even the quest 2 passthrough with it stitching multiple images together and being black and white is still pretty decent and easy to get used to IMO. If quest passthrough was color and a bit higher resolulution i feel like i could do everything i need to im my daily life in passthrough.
The issue is there really isn’t any consumer AR to try yet. Only things that startups show off. But I have a feeling he might not have tried the quest 2 passthrough
quest 2 passthrough is terrible, grayscale and very blurry, hard to use for more than a few minutes if one has to actually look at the real world for too long. If any AR device tried to enter the market with Quest 2's passthrough it would be crushed.
@@Wobbothe3rd yea I know it's blurry but I was addressing the alignment issue that was mentioned in the video. I've never experienced any of the uncomfortable side effects he mentioned from passthrough. The blurryness obviously becomes uncomfortable when you're trying to read something small like your phone but if the camera resolution were increased, I could see it being fairly comfortable to use
Technically all AR, even phone-based AR, already can be useful for some things. Hololens is already used for work. The problem is passing that threshold of positive value for many millions of consumers instead of being a niche device. The founder of Oculus has a blog post claiming that current VR is still not good enough for many people even if it was completely free, especially for long time use. Big corporations like Meta and Apple aren't chasing useful AR, they are chasing better-than-smartphone AR for billion+ people. And that's the point of this video - it won't be possible even on a decade. On the other had some people were happy with HMDs that we had in the 90s. It's a spectrum.
Only 9 minutes into it and I already think that Passthrough AR will be never a solution.
Great video but please fix the audio dropout in the next one.
uh, motion prediction vectors to match real life and the overlay ar graphics
"The next apple product will be a failure". Famous last words!
A little bit of a buzz kill but good interview none the less.
이 정도는 되어야 전문가가 말하는 전문지식이라고 할 수 있지..
Great interview. Learned a lot.
One thing I think is an unfair criticism of passthrough VR is that you put the image all on one focal plane. I don't think that's going to be true for long with varifocal displays like the half-dome prototypes shown by Meta recently.
I also think the solid state varifocal field is definitely progressing very well
I think this guy doesn't really understand a lot of the use cases people will use AR for. cuz like he wants to make this perfect AR that mixes the augmented and real world perfectly. while I think a lot of people just want to do VR stuff but they also want to be able to see the world around them a bit. like productivity stuff where you would just want a lot of large screens or other assets floating around you while also seeing a real mouse and keyboard to control them with. if you use that kind of stuff for your job you don't want to be stuck in VR all day while you're sitting in your office. it would be nice if you could look past your screen to see the door and shit. or if you're using it to model stuff in 3d or something, again you don't really need a full VR world surrounding you and being able to see the office (and possibly co-workers working in the same space as you) would be very nice. even if they don't look like picture perfect representations of the real world....
Binoculars
Easy ... just remove the eyes and replace them with high tech AR balls.
The constant audio dropouts are incredibly disruptive. I wish the guy had a decent audio setup. :(
I think the future of AR isn't gaming but instead displays for phones and media. I can't tell you how many times a day I see someone awkwardly supporting their arm to watch a clip or movies on their phones. If you can offer people a nice screen behind a set of nice looking sunglasses without a wire, you have a product. Its coming.. Its almost here with things like the Nreal Air. But its a few years away.
It's about time some of these "AR >>> VR, fancy glasses or bust, nobody wants VR!!!" companies are getting some cold water thrown at them. Sorry dawgs, you'll need to grit your teeth and focus on passthrough like everyone else. No shortcuts!
When AR fails over the next few years, do you think it will significantly harm the progress of VR?
If apple fails in their showing: it will be painful
@@SadlyItsBradley I think the use cases presented in the AR hype are, in reality, virtually non-existent. I was a VR early adopter with a Z-800, later the DK1 and so on. With well over a dozen HMDs so far (including a HoloLens 1) I think it's fair to say that I'm a fan of it. Some seem to think that AR is the next evolution of VR, but I don't see it that way. I know there are some very cool things you can do with AR, but I really have very little interest in it when compared to what can be done in VR. It reminds me a little of the marketing hype around 3DTV. The marketing wanks hyped it as "the new way you'll watch TV" when they should have been selling it as an _additional capability_ (and an extremely cool one IMO) that you can use now and then for specific things. I see AR as a nifty additional capability for HMDs, but I would have zero interest in an AR only device.
Karl is going to get pretty surprised.
Never underestimate what can be done now that we have insane neural nets and compute to boot.
The same thing was pretty much said for inside out tracking. It will never work, no one will code a decent enough solution. Half write. Humans dont write any of that code anymore, its all handled by neural nets, and no human would of been able to code from scratch the same solutions.
Working voice to text ? Never going to happen, the words of the top tier people working in the field 15 years ago. They were half right, once again. No human is going to code an AI that can do extremely perfect audio to text, its done by Neural Nets.
In the end, AR and anywhere-computing will be ubiquitous. I like being immersed in movies and games on a Friday night, but I use my computer (and phone) for obscene amounts of time every day. The "shooty games on the wall" Karl mentioned applies to AR as much as Doom does to modern computing.
I wonder though, which side of "the great filter" are the perfect AR glasses on? Seems like we got a pristine world and wrecked it in less than 100 years.
I agree I think passthrough AR is going to be garbage. Apple or others.
The problem with VR, AR, MR, XR, ZR(?) is the R part.
It seems the companies working on these projects do not have...
(for the lack of a better description)
A classically, hard core, extremely well-trained Physicist in their teams.
A person to point out and explain that there is no Real R... with glasses or not...
Trying to put together a piece puzzle, with pieces from 3 different boxes...
Yeah one will find pieces that seem to have "problems", occlusion/paralax/delays/limitiations to dynamic range/resolution....
And they waste time trying to fit pieces from different boxes.
Obviously it is impossible to say what a Physicist would say to help with all this in a youtube video,
but here's something more than just the thought experiment above...
Go find a colour-blind person and try to talk about their view of reality. AND then try to solve that "problem" :)
Wonderful series and interview Brad! Keep up the good work, and waiting on a deep dive of Nreal's Steam thingy :)
sure not everyone will use a ar headset. does everyone has a gaiming pc no. its ok to not apeal toeverybody. imagine the same discusion but with the topic gaming pcs. "its to expensive for kids" "old people will not use it" "the grafic dosnt look 100% fhotorealistic" " oh you can see a pixel on the screen" . it dosnt have to pe 100% perfect. no product is. and not every prodact is for everyone. ther are so many people in the world and we only think about who will not use the device. Gaming was never perfect. In the begining aming was realy nesh. its better to grow naturaly to a bigger audience. many think that it will change the world.but if the expectatioons are so hige. ther is no wonder that somone will be disapointed.
Yes. MR will be just that - geeks, gamers, enthusiast and pro users. But Meta wants billion+ users and replace smartphones. That's a whole different challenge that requires 100x better tech.
@@kazioo2 I think meta is the only one who wants billion users. Gaming pcs / consols are also a big industry. Not every gamer is a neard. But I get whee u coming from.
@@kazioo2 At least their unrealistic expectations have kickstarted things for others with more realistic ones, though~
08:09 omg, he really thinks that photon-> electron -> photon delay can be noticeable by a human?
can't you "just" fix the camera placement thing with reprojection or something? like the quest 2 doesn't have camera's near your eyes at all but the passthrough, while far from perfect, doesn't make me feel weird at all. I can imagine some better version of that in color working decently enough. like ofcourse you won't be fooled into thinking you're looking at the real world but if you can just see the world around you and your keyboard and shit properly it would solve a lot of AR issues already.
like this dude seems to be focusing way too much on making the real world look good in AR while IMO you should focus on making the AR content look good cuz that's why you're wearing the thing in the first place. you won't be focusing on the background 99% of the time. so I think passthrough AR is the way forward for the very near future cuz at least there the AR stuff doesn't look like some blurry hard to see ghost image....
All the passthrough AR problems have clear solutions, while optical AR just as us to believe there is a technological apth towards it.
if u ask Karl vr shouldn't exist at this quality according to him ...as far as I remember ....or maybe his opinion has changed
Last I asked, he said he had a Quest 2 and only used it for 20 minutes because he got nauseous.
@@SadlyItsBradley That explains a lot. VR as it is can provide all kinds of truly amazing immersive experiences and it's only getting better. I have exactly zero interest in AR, however.
VR is sucessful but actually not that big yet. More skepticism and criticism is warranted, even with the Quest 2 selling 10 million plus. VR still has a LONG way go to live up to the hype, and standalone has big tradeoffs compared to PCVR despite the benefits. Karl may turn out to be right in many ways, the jury's still out,so to speak.
So many problems with getting AR to work well, what a shame.... Maybe everyone should just abandon this AR stuff for now and focus on VR entirely.
Who let the party pooper in.