This is insane. I love your guys' willingness to break some new ground and explore what's possible with today's tech. You're literally setting the precedent!
That's the right attitude. I expect a lot of "why would you do this?" comments, but you're point about a willingness to explore is what leads to new ideas and opportunities. Thanks for watching!
Using a monitor in MR vs a monitor IRL isn't going to change things if you stop and think about it. Better to ask yourself what does MR exclusively unlock? Spatial information. For example, the MR user could be given the superpower to see a dynamically tracked lens frustum, focus plane and DOF range all virtually projecting out of the cinema camera as it moves. This would make pulling focus, even within the most challenging scenarios much more intuitive.
As a boom op this would also be brilliant. Not falling over monitors on a tight set, or having to glance at an iphone to check framing would be a godsend.
I never thought of that! But the notion of other crew members having access to their own wireless monitor is a brilliant use case. The cost of a monitor + wireless send is a lot more than a Vision Pro. And I love the idea of audio being able to see the set (and booms, etc) without having to ask village. Very cool idea!
Terradek's aren't cheap even streaming to an iphone. Plus I've noticed people getting picky about who gets ON the monitor channel. We've been having people sitting outside the studio in a van with Terradeks hoping for a peek.@@Michael_Cioni
It’s pretty impossible to wear this device long term, it’s too heavy and all of the weight rests on your face. I’ve spent additional 500$ on top of the 4K to find a comfortable way of wearing it, and unfortunately it does not exist. Tried all the light shields that could fit, top straps, multi straps, forehead resters, modding other headsets and results are always the same, being uncomfortable and in pain. The AVP grinds your face, unfortunately. The AVP is fragile, so you def would have to own 2-3 three headsets for redundancy. I also work in sound on set and post and I’ve struggled to incorporate into any of my workflows. It sounds great on paper, HQ monitor strapped to your head. I would like to see the 1st ACs face after a real full set week. They would look horrible, possible bruised and their eyes would be blood shot. It causes major eye strain as well. It’s nowhere near a device that could be used in the camera department. “You’re too close to an object” messages would constantly pop up moving on set, I get those just sitting around with AVP. The pass through is ok in good lightning but we often work at night and pitch black darkness and the AVP is not capable of being used in those conditions either. First it would severely limit your mobility, secondly it would constantly disconnect saying tracking has failed. While there is no doubt at some point in the future this will be possible with a different iteration of AVP we are not there yet. It needs the ability to hardwire Ethernet, usb-c etc for real professional use. I have the developers kit and the USB connection is very limited at this point. I couldn’t make it through this video fully due to the presenter shout-talking so maybe they raised some of these points. Like what if the AC is wearing glasses, issues arise with distortions etc. Another thing that made me laugh was that they had to shoot this on an iPhone to get it to work.
The Immersed Visor headset coming soon might be more appropriate, high res pass through and robust tracking is all you need really. Though I believe that’s not standalone. The poor performance in low light would be a big problem. How you solve that without having bigger sensors/lenses is an interesting thought.
I do cybersecurity for a living, I've been CS engineer my whole life, I only take photos with one of the small sony cameras and for some reason I love your videos, series, and all the creative insight you give us. I'll probably never shoot a movie on an iphone, or pull focus with AR but thank you for showing us what is possible and how is done
Yeah there's just absolutely no way this is fast enough for pulling focus on pro shoots where the AC has to be accurate the first time and the camera is moving.
Really cool idea and as a focus puller myself, the first thing I thought when the vision pro was announced was focus pulling. But in practice, this seems like it gives you exactly what a nice monitor on a stand gives you except with more failure points. There also doesn’t seem to be a zero latency solution for image transition on any camera system. Then you have to worry about battery life. The reported battery life on the system would barely get you through a scene much less a 10 - 12 hour day. Cool idea. Nice to see it tested. But not viable in a professional setting.
All good points. Definitely too early for this to be widely adopted in everyday productions. But if we want to see tech like this become more mature and meet all our needs at scale, we need to try it, test it, and provide feedback to manufacturers so they can improve the ecosystem. I expect a lot to change in this space over the next 10 years.
Mike, I met you in 2010 when you were doing Light Iron Digital. You gave talks when RED was having in-house events in Hollywood. I love that you're still going hard on innovating filmmaking tech. I salute you, chief.
Holy shazang!!!! I dabble in video making but this is next level mind blowing! I found your channel about a day ago, and I'm hooked! Apple and off the shelf products that you can buy means anyone can make their own movie (albeit not totally big budget film, but hey, less than big budget can win Oscars) and puts the little film companies in the mix! You are breaking the mold and I'm so here for it!
Another awesome experiment pushing the limit what's possible! Would love to see how you'd set this up if you flipped the experiment - focus pull with the Vision Pro on a pro set with (mostly) unlimited budget. I could also see a reality soon where the lidar from the camera tracks a mark you set and shows a realtime distance line in AR
I'm a film fan and alway love seeing behind the scenes videos but I'd never consider being part of a production. Still, your videos are so well produced and I really like seeing and hearing about all the different tools and apps you use. Also, I really appreciate that you don't use multiple takes when you're speaking to camera, I always find that technique really annoying! Can't wait for the next video 👍
As a focus puller of 15 years feature film level experience I can tell you this woman isn’t what most on the industry would consider a focus puller. In future please find feature level people to test this kind of stuff not film student level. The fact she’s pulling of a laptop screen not looking at the action but essentially away from set is a massive red flag. No professional focus pullers would be caught dead using this system.
This was an amazing video, informative, entertaining, and creative. I think this was the first video from this channel I’ve seen and now I’m subscribed.
Apple Vision Pro has amazing eye tracking, the best integration would be to sync that feature with the focus-pulling wireless device, so wherever you look, it focuses. That should come very soon.
Amazing work 🎉 it’s so fun seeing the process and knowing that even professionals want to use these technologies or even acknowledge them as potential tools on set. Will be following this journey long into the future. Can’t wait for what’s next!
The channel is no joke, you are like Mojo Jojo but in the world of cameras and tech, you pull something new out of the bag like a surprise, like a magician.
This is absolutely amazing, thank you for making this project and sharing it! It’s what I’ve been searching for. I’ve been looking for a solution very similar to this for live video into a Vision Pro. My next hurdle to solve is inputting multiple live video feeds to the headset from different types cameras (at least two.)
Really cool idea, I work as a first I wouldn't mind this but there seems to be some limitations. Personally, latency has to be extremely minimal. So if industry video transmission companies, like Terradek, get involved then I would be game. Also, I move around a lot. Carrying a big Arri or Venice is already really heavy. Still, I can see this being the future, just have to work out the kinks
Every cinema camera from lens to sensor to image processor to wireless transmitter has some latency. We found this felt to be just about the same. If you put on a VP they are about 12ms latency (imperceptible). The APower Mirror app presented a result that was probably around 300ms, but that includes the delay of the camera. So it was totally useable for focus pulling with room to improve.
We didn't detect any noticeable latency between the Nucleus and the Vision Pro. All cameras to wireless transmissions have some latency, but we've done this with professional tools thousands of times and honestly it felt like the latency to Vision Pro was similar to Alexa or RED or Sony. It didn't impact out shooting. The weight and Timecode were bigger problems. Thanks for the question!
Me too! We're going to NAB in 2 weeks and I'll be speaking with wireless transmission companies to see what they are thinking about in terms of adding an OS solution to the portfolio so Vision Pro can be a recipient of their signals.
This whole thing is cool but it’s really amazing that what the iPhone needed was a good camera app. Black Magic really knocked it out of the park with their app
You know I don't think I heard anyone put it that way before...but I think you're 100% right! The BM app really unlocked a ton of value that the phone was capable of delivering...but never effectively delivered. Great point! And now we shoot with it for socials every day and built a nice custom LUT to help it look...less Apple-ish. Thanks for the comment:)
two things I'd like to know/add: high resolution display: yes the vision pro has one, but at what resolution does the stream come in and what is the latency of it? And also how do you get the streams from other cameras into the VP or is it just for iPhones?
Great question! The Vision Pro is capable of display much GREATER than 4K, and the tool we used APower Mirror says it was capable of 4K, but we weren't sure if we were actually getting that. So my guess is there will be major innovation opportunities to ensure 4K transmissions to VP (much like watching a movie off AppleTV+ or Netflix on VP). So the technology works wirelessly and only needs to be integrated in iPhone mirroring tools (and subsequently other wireless transmission tools). thanks for the comment!
Batteries on both Vision Pro and iPhone 15 are something you have to factor in. So we bought mobile USBC battery packs that could live on or near devices that needed frequent charging.
As a 1AC, this is definitely interesting. Where the tech is right now it's not realistic at all for professional productions with the added latency, reliance on iOS and lots of extra points of failure. That being said, when VR/AR headsets get smaller and this tech improves, it might be a viable option for some productions
I agree. This is the "top of the 1st inning" so innovation will be rapidly expanding over the next few years in this field. However, of all the limitations and issues we had to problem-solve, latency wasn't one of them. We found that the latency was around 200-400ms from lens to village, which is pretty similar to an Alexa > Teradek > Monitor.
imagine projecting a colored plane of focus and field of view on the camera itself, using lens metadata and a tracker on the camera, the vision pro could project a plane in its 3d scan of the environment so the focus puller doesn't even need to see a camera feed at all. If everyone else on set could see the cameras field of vision in a cone, it'll also be really easy for grips, gaffers and soundies to keep booms and lights out of the frame, art dep could focus in real close on placing props on effective spots, completely removing the guess and check that typically happens with the crew and whoever is viewing a monitor.
You guys do know the sync e doesnt need that cable to work w bm camera right… it pulls it directly over bluetooth in the same fashion of how the sync e’s talk to each other. no cable needed.
maybe focuspull but you normally need to see the markings on your remote and also, as she handled the camera while wearing the VP, this wouldn't be possible in VR, only AR let's you interact with your surroundings
4:30 vision pro can at most display 100nits brightnes. Also you view it from so close that it is not enough resolution to not see individual pixels. And you are better off looking at a phone or a real monitor.
It’s a great use case for using Vision Pro, imagine when developers start making proper tools for film makers on Vision Pro. The only thing I would say is this is definitely mixed reality and not true AR. I think you couldn’t ever get this good with true AR, no matter the form factor. High quality displays in what is basically a mixed reality headset is the only way moving forward and with slimmer design and a broader field of view, it’s exciting to see what’s next.
We found that there wasn't any noticeable lag from camera to headset. There's always a little latency between any lens, sensor, video signal and wireless transmitter, but we felt this was about the same as a normal cinema camera and didn't impact our focus pulling. There are other stability issues with timecode, the app and the weight of the device, but we were surprised latency wasn't an issue!
@@Eddie32791 oh I do know my brother! I have a WCU-4 and Nucleus out in the rental circuit .I just admired but could not justify paying for the vision pro but this swings it in my favour
That looks pretty fascinating. However, there are still some problems that need to be resolved. As a camera assistant myself, I always have to be able to rely on the equipment, and that doesn't seem to be the case here yet. Relying on third-party apps can really cause problems on set. In addition, streaming via WIFI and HDMI creates a latency problem, and as a 1.AC every millisecond counts. But nonetheless, I find this kind of experiment really enjoyable and hope that it might become usable at some point.
I mean, very interesting concept but i think that there should be a lot more to the apple vision pro than just creating a virtual monitor somewhere on set. If you want more mobility you can rig a 7" directly on the hand unit of the focus kit and walk around with that and I think that would be cheaper (and safer) than having someone with weird goggles on set all day. Also an awful lot of focus puller (maybe more seasoned ones, but everybody should work this way) prefer to use the monitor as a focus checking tool and pull focus by measuring distances, using marks and just seeing distances by eye, it's a bit more difficult and you're basically pulling without seeing the image but in this way you can be always aware of where the camera is in relation with the actors. If someone miss their mark if you are just looking off from the monitor you are not aware of that until you see the picture out of focus and in that case there is nothing you can do, there is an out of focus part in the shot. I think that if we want rally to implement the Vision Pro on set we should use it to create something that it would be otherwise impossible, maybe combining the lidar scanning, mapping the lenses and the camera so that you can see the frostrum of the lens, the current focus position in AR, the minimum focus distance, maybe you could be able to add marks. Also you could have camera infos floating around without being relegated to a rectangle somewhere. I don't want to criticise your work and your video, I know that something like I've described doesn't exist yet and the vision pro just came out so everything it's new, but your video made me think a little bit about that
If black magic comes with the apple vision pro app to seamlessly control the video rig, next gen vision pro will sell like hot cakes(if the price is affordable) 😅.
Hi Peter! The stuff of me in the office is shot on a Panavision DXL2 using PanaSpeed lenses (a 40mm to be exact). I have a custom LUT developed and use a Creamsource Vortex 8 as my key light and Aputure Infinibars for a backlight and rear wall light.
In terms of the actual future, I think the Vision Pro is the camera; computational photography simulates cameras and lenses; depth mapping recorded for VFX and of course, the VisionPro viewer of the movie can take over eye-tracking focus from the camera operator. Other spatial camera devices will make positioning and moving the camera possible, so you aren't tied to first person point of view
Great effort but you didn’t realize a major issue with the Vision Pro …latency. Until this gets resolved I highly doubt the Vision Pro will be used to pull focus remotely on pro shoots.
We were surprised, but didn't experience noticeable latency that was more than any normal cinema configuration. Felt very much the same in terms of latency (probably somewhere around 300ms give or take in my estimation).
@@Michael_CioniYou seeing latency vs a professional focus puller in a shallow DOF situation are two different things. Plenty of people who use “low latency” wireless image transistion devices don’t think they see any latency. But it can make or break a shot extremely easily. There is a reason why Teradek is the gold standard in the industry
Latency is the big thing for me in this situation. Using a wifi based system is the worst possible option when trying to fight latency. That combined with the inherant latency and inaccuracy from the nucleus is a recipe for disaster on tight shots.
Indeed. I agree with you. I have several Teradek Bolts. To clarify what I'm referring to is the innate latency between a lens, a sensor, an image processing pipeline, a video output signal and the wireless transmission. Even tho Teradek's are marketed as zero latency, there is latency before they are given a signal. In other words, it takes a little time to get light from a lens to an any monitor, and 200-300ms is fairly common. Only film cameras with optical viewfinders (and video taps) are able to avoid this. I agree with you that poor latency can absolutely blow a shot. But whether they know it or not, the professional community has become accustomed to acceptable latency due to digital camera image processing pipelines. I'm intimately familiar with professional camera systems and monitoring latency and from my experience, this very early iteration of AVP wireless monitoring is on par with the latency of traditional cinema systems. Get your hands on an AVP and try it for yourself and LMK if you agree. Thanks for the comment!
I’m not aware of any CA artifacts. I own a set and watch Dolby Vision films that I worked on (so I know the content well on a SONY X300 or 310). I’m not saying there isn’t CA artifacts in the Vision Pro Display, but I’ve not personally seen any. The films I’ve helped color correct look spot-on to me. But since you raised this point, I’m gonna ask around and see what my colorist colleagues say. Thanks for the comment!
Why would you need it And move quickly? I can move quicker without this on my head which is destracting my eyesight and range of view plus not to mention my perception and instinct of how someone else near me is going to move And relatively cheap? If you can't do it on a high end monitor then get another job Are they paying you?
Thanks for the comment. To be clear, no one is paying us:) I've been working on high-end cinema for many years and what helped me develop as both a filmmaker and entrepreneur is to remain curious about the intersection of art and technology. Your points about mobility are valid, which is why it's important to remember AVP is extremely new and AR integrations in the production space are almost totally nonexistent. But curiosity, testing, and user feedback will help fuel innovation in this space that will provide solutions and opportunities that may seem foggy today. This has always been the case with new technology in the production space, but eventually, fads become trends and trends become standards. Appreciate the feedback!
Finally a video about the AVP worth watching. Thanks for seeing the possibilities, and enabling capabilities instead of just being another whiner complaining about what the AVP isn't.
Hey! Thanks! Glad this was fun to watch and informative! I always look at things from the perspective of glass-half-full. And with brand new technology, it's all about getting excited about what's possible. Thanks for the comment:)
Not really AR... Apple promotes it as "IMMERSIVE Reality," because the user isn't taken out of the environment. Nitpick, I know... but I was deeply impressed with how you've taken a very basic visionOS release and incorporated it into your workflow. As an actor, it'd also be helpful for us to see the directors vision on a shot, so we'd know exactly how to act for camera (for those directors that use a video village). Overall, very very impressive!!!
The latency on this must be terrible. I personally put a 12.9inch iPad on a chest rig as a back up monitor to a real focus pulling monitor. I send a latency free signal to the focus monitor and then go out from there to an Accsoon cineview SE. it gives me a more mobile monitor and gives a signal to other departments who want a feed for their smart devices. I wear my nucles on my right leg on a leg gun holster.
It surprised us that there was hardly any noticeable latency added by the APower Mirror (app) wireless transmission from the iPhone (running Blackmagic Camera App). Had we had that issue, we would have brought it up here along with other issues we pointed out (timecode $ weight). Today this setup did work, but it’s not viable this early. That’s why feedback from professionals like yourself to relevant manufacturers is how improvements will be deployed. Thanks for the comment.
I may be in the minority here, but I have no idea what “focus pulling” meant and was lost most of the video. Just thought I’d give some feedback to improve your videos in the future.
It's not a video for beginners, it's showing other professionals what they can do. Googling 'focus pulling' takes 5 seconds, way faster than writing your comment.
I like the part where you never talked about how terrible the latency would be using this in the real world. It's just a gimmicky click-bait video with no actual professional application pulling focus on set.
We actually spent a lot of time testing lots of things and the Vision Pro did not exhibit any latency issues that felt different from that of a normal cinema solution. Normal cameras have 200-400ms latency from lens to sensor to processor to wireless transmitter/receiver. Having done that thousands of times, we felt that iPhone to Vision Pro was probably in the 300ms range, give or take (I didn't actually measure it with a hardware tool). But it was interesting that of all that areas for innovation and problem solving, we were really surprised latency wasn't one of them! Thanks for the comment:)
This is insane. I love your guys' willingness to break some new ground and explore what's possible with today's tech. You're literally setting the precedent!
That's the right attitude. I expect a lot of "why would you do this?" comments, but you're point about a willingness to explore is what leads to new ideas and opportunities. Thanks for watching!
@@Michael_Cioni can you do this on the quest, since that will bring in more users.
Using a monitor in MR vs a monitor IRL isn't going to change things if you stop and think about it.
Better to ask yourself what does MR exclusively unlock? Spatial information. For example, the MR user could be given the superpower to see a dynamically tracked lens frustum, focus plane and DOF range all virtually projecting out of the cinema camera as it moves. This would make pulling focus, even within the most challenging scenarios much more intuitive.
I like the way you think
As a boom op this would also be brilliant. Not falling over monitors on a tight set, or having to glance at an iphone to check framing would be a godsend.
I never thought of that! But the notion of other crew members having access to their own wireless monitor is a brilliant use case. The cost of a monitor + wireless send is a lot more than a Vision Pro. And I love the idea of audio being able to see the set (and booms, etc) without having to ask village. Very cool idea!
Terradek's aren't cheap even streaming to an iphone. Plus I've noticed people getting picky about who gets ON the monitor channel. We've been having people sitting outside the studio in a van with Terradeks hoping for a peek.@@Michael_Cioni
It’s pretty impossible to wear this device long term, it’s too heavy and all of the weight rests on your face.
I’ve spent additional 500$ on top of the 4K to find a comfortable way of wearing it, and unfortunately it does not exist.
Tried all the light shields that could fit, top straps, multi straps, forehead resters, modding other headsets and results are always the same, being uncomfortable and in pain. The AVP grinds your face, unfortunately.
The AVP is fragile, so you def would have to own 2-3 three headsets for redundancy.
I also work in sound on set and post and I’ve struggled to incorporate into any of my workflows.
It sounds great on paper, HQ monitor strapped to your head.
I would like to see the 1st ACs face after a real full set week. They would look horrible, possible bruised and their eyes would be blood shot. It causes major eye strain as well.
It’s nowhere near a device that could be used in the camera department.
“You’re too close to an object” messages would constantly pop up moving on set, I get those just sitting around with AVP.
The pass through is ok in good lightning but we often work at night and pitch black darkness and the AVP is not capable of being used in those conditions either. First it would severely limit your mobility, secondly it would constantly disconnect saying tracking has failed.
While there is no doubt at some point in the future this will be possible with a different iteration of AVP we are not there yet.
It needs the ability to hardwire Ethernet, usb-c etc for real professional use.
I have the developers kit and the USB connection is very limited at this point.
I couldn’t make it through this video fully due to the presenter shout-talking so maybe they raised some of these points.
Like what if the AC is wearing glasses, issues arise with distortions etc.
Another thing that made me laugh was that they had to shoot this on an iPhone to get it to work.
The Immersed Visor headset coming soon might be more appropriate, high res pass through and robust tracking is all you need really. Though I believe that’s not standalone.
The poor performance in low light would be a big problem. How you solve that without having bigger sensors/lenses is an interesting thought.
@@longwelshnight vision lense
The first thing I tested day one was connecting Vision Pro to my Komodo X via Red Control. Excellent post.
The quality of this video is absolutely bonkers! Instant sub, keep up the fantastic work
Apple should have this video on their page! Great video! Professionally made!
damn this video was incredible. great work to everyone involved!
I do cybersecurity for a living, I've been CS engineer my whole life, I only take photos with one of the small sony cameras and for some reason I love your videos, series, and all the creative insight you give us. I'll probably never shoot a movie on an iphone, or pull focus with AR but thank you for showing us what is possible and how is done
Has anyone mentioned the latency on the Vision Pro? I assume it’s not as fast as a Teradek wireless system
My first thought too, latency 🫨
Asking the right questions
Yeah there's just absolutely no way this is fast enough for pulling focus on pro shoots where the AC has to be accurate the first time and the camera is moving.
The apple vision pro is about 4 miliseconds of delay, while the screen casting app has delay, using a macbook will make it more like 5 miliseconds.
The weight is also a factor not to mention on hotter sets, it will get pretty sweaty. But there is potential.
Really cool idea and as a focus puller myself, the first thing I thought when the vision pro was announced was focus pulling.
But in practice, this seems like it gives you exactly what a nice monitor on a stand gives you except with more failure points. There also doesn’t seem to be a zero latency solution for image transition on any camera system. Then you have to worry about battery life. The reported battery life on the system would barely get you through a scene much less a 10 - 12 hour day.
Cool idea. Nice to see it tested. But not viable in a professional setting.
All good points. Definitely too early for this to be widely adopted in everyday productions. But if we want to see tech like this become more mature and meet all our needs at scale, we need to try it, test it, and provide feedback to manufacturers so they can improve the ecosystem. I expect a lot to change in this space over the next 10 years.
@@Michael_Cioni oh 100%. It needs to be tried and tested. I’m happy it was!
Mike, I met you in 2010 when you were doing Light Iron Digital. You gave talks when RED was having in-house events in Hollywood. I love that you're still going hard on innovating filmmaking tech. I salute you, chief.
thank you! Stay curious:)
Holy shazang!!!! I dabble in video making but this is next level mind blowing! I found your channel about a day ago, and I'm hooked! Apple and off the shelf products that you can buy means anyone can make their own movie (albeit not totally big budget film, but hey, less than big budget can win Oscars) and puts the little film companies in the mix! You are breaking the mold and I'm so here for it!
I can't begin to explain how well put together this video is.
Wow! Thanks! Check out Episode 23 and other episodes to learn more! Thanks for the comment;)
Atlas mercurys being pulled by a vision pro is MADNESS.
I love it
Another awesome experiment pushing the limit what's possible!
Would love to see how you'd set this up if you flipped the experiment - focus pull with the Vision Pro on a pro set with (mostly) unlimited budget. I could also see a reality soon where the lidar from the camera tracks a mark you set and shows a realtime distance line in AR
Not a bad idea. Whose budget?? ;-P
@@Michael_Cioni Panavision 👀
Awesome and criminally underrated stuff here guys!
"Criminally underrated" - that's funny. I'll take it, tho:) Thanks for the comment!
I'm a film fan and alway love seeing behind the scenes videos but I'd never consider being part of a production. Still, your videos are so well produced and I really like seeing and hearing about all the different tools and apps you use. Also, I really appreciate that you don't use multiple takes when you're speaking to camera, I always find that technique really annoying! Can't wait for the next video 👍
As a focus puller of 15 years feature film level experience I can tell you this woman isn’t what most on the industry would consider a focus puller.
In future please find feature level people to test this kind of stuff not film student level.
The fact she’s pulling of a laptop screen not looking at the action but essentially away from set is a massive red flag. No professional focus pullers would be caught dead using this system.
This was an amazing video, informative, entertaining, and creative. I think this was the first video from this channel I’ve seen and now I’m subscribed.
That is so great to hear! Thanks for following us and sharing this comment! Really appreciate it:)
My first impression of this video was WOW! Brilliant production, thank you for posting.
Apple Vision Pro has amazing eye tracking, the best integration would be to sync that feature with the focus-pulling wireless device, so wherever you look, it focuses. That should come very soon.
Amazing work 🎉 it’s so fun seeing the process and knowing that even professionals want to use these technologies or even acknowledge them as potential tools on set. Will be following this journey long into the future. Can’t wait for what’s next!
The channel is no joke, you are like Mojo Jojo but in the world of cameras and tech, you pull something new out of the bag like a surprise, like a magician.
Nice idea, but surely at this stage the resolution and lag is going to be way off for accurate focus
This would be awesome for me to use as a boom op as well. Getting to see the exact letterbox and how much I can push it!
Amazing content! This is the future 🔥
I really love your videos. Storytelling, visuals, sympathy, everything is mint. Well done. Seriously.
It'll be a great idea to try.
This is great! I really hope to see some integration with teradek tho
I’ll try this on my show. A decent amount of location work with tons of CG elements
Excellent video!!!! I just got mine and I see the potential of this to replace so many things it is unreal.
This is absolutely amazing, thank you for making this project and sharing it! It’s what I’ve been searching for. I’ve been looking for a solution very similar to this for live video into a Vision Pro. My next hurdle to solve is inputting multiple live video feeds to the headset from different types cameras (at least two.)
Im sitting here trying to figure out why Strada doesn't have millions of subscribers.
fr man
Because they are not charismatic I think??
Because this guy is annoying AF and didn't ask any of the right questions as to wether this is really a viable solution or not.
Amazing. Now only to see it used on cinema cameras. And if this setup allows to pull focus more precisely than regular video village
No way. Latency
Pretty cool idea. Now I want to try it out.
Really cool idea, I work as a first I wouldn't mind this but there seems to be some limitations. Personally, latency has to be extremely minimal. So if industry video transmission companies, like Terradek, get involved then I would be game. Also, I move around a lot. Carrying a big Arri or Venice is already really heavy. Still, I can see this being the future, just have to work out the kinks
This is wild. Love it!
My head blown out 😲. Interesting ❤
Low Latency? Not sure about that over that Wifi App... What was the delay like?
Every cinema camera from lens to sensor to image processor to wireless transmitter has some latency. We found this felt to be just about the same.
If you put on a VP they are about 12ms latency (imperceptible). The APower Mirror app presented a result that was probably around 300ms, but that includes the delay of the camera. So it was totally useable for focus pulling with room to improve.
How about the delay from apple pro vision to núcleos?
We didn't detect any noticeable latency between the Nucleus and the Vision Pro. All cameras to wireless transmissions have some latency, but we've done this with professional tools thousands of times and honestly it felt like the latency to Vision Pro was similar to Alexa or RED or Sony. It didn't impact out shooting. The weight and Timecode were bigger problems. Thanks for the question!
Very good video.
very good
great video. excellent work
Criminally underrated channel. So much money and hard work behind this channel and it really shines through
Well done, mate!
Thank you!! Check out Episode 23 and see how we did the anamorphic shoot!
Finally i see something useful. Cudos
This is promising . I definitely would love to see this tried with a proper cinema camera.
Me too! We're going to NAB in 2 weeks and I'll be speaking with wireless transmission companies to see what they are thinking about in terms of adding an OS solution to the portfolio so Vision Pro can be a recipient of their signals.
@@Michael_Cioni that would be game changing if they did. I’m on the fence about getting one soon .
Dude...I've been working on this!!!
This is genius
This whole thing is cool but it’s really amazing that what the iPhone needed was a good camera app. Black Magic really knocked it out of the park with their app
You know I don't think I heard anyone put it that way before...but I think you're 100% right! The BM app really unlocked a ton of value that the phone was capable of delivering...but never effectively delivered. Great point! And now we shoot with it for socials every day and built a nice custom LUT to help it look...less Apple-ish. Thanks for the comment:)
Cool idea like to try the job
I’d presume 6:55 you’d hdmi from iPhone usb-c tho to share normally?
8:46 ah
that was damn interesting. subscribed
Amazing insight(s). Thanks and kudos to your team for trying new technologies. Subscribed!
Very interesting video.
Awesome and criminally underrated
You can use monitor control in Sony cameras probably. That is interesting
Lads, did you just put the EF-PL adapter on the Beastgrip DOF MK3?
Fantastic proof of concept. Early adopters & experimentalists are awesome 👍
simply amazing
two things I'd like to know/add: high resolution display: yes the vision pro has one, but at what resolution does the stream come in and what is the latency of it? And also how do you get the streams from other cameras into the VP or is it just for iPhones?
Great question! The Vision Pro is capable of display much GREATER than 4K, and the tool we used APower Mirror says it was capable of 4K, but we weren't sure if we were actually getting that. So my guess is there will be major innovation opportunities to ensure 4K transmissions to VP (much like watching a movie off AppleTV+ or Netflix on VP). So the technology works wirelessly and only needs to be integrated in iPhone mirroring tools (and subsequently other wireless transmission tools). thanks for the comment!
This was so cool! How did you work around the Apple Vision Pro's limited battery life??
Batteries on both Vision Pro and iPhone 15 are something you have to factor in. So we bought mobile USBC battery packs that could live on or near devices that needed frequent charging.
As a 1AC, this is definitely interesting. Where the tech is right now it's not realistic at all for professional productions with the added latency, reliance on iOS and lots of extra points of failure. That being said, when VR/AR headsets get smaller and this tech improves, it might be a viable option for some productions
I agree. This is the "top of the 1st inning" so innovation will be rapidly expanding over the next few years in this field. However, of all the limitations and issues we had to problem-solve, latency wasn't one of them. We found that the latency was around 200-400ms from lens to village, which is pretty similar to an Alexa > Teradek > Monitor.
great video
What adapter did you use to mount the cine glass on the iphone? Or was i missing something?
Cool video.
Great question! Check out Episode #23 and we go into details about the glass and custom adaptors!
Super interesting!
imagine projecting a colored plane of focus and field of view on the camera itself, using lens metadata and a tracker on the camera, the vision pro could project a plane in its 3d scan of the environment so the focus puller doesn't even need to see a camera feed at all. If everyone else on set could see the cameras field of vision in a cone, it'll also be really easy for grips, gaffers and soundies to keep booms and lights out of the frame, art dep could focus in real close on placing props on effective spots, completely removing the guess and check that typically happens with the crew and whoever is viewing a monitor.
Wow. Killer idea.
This is basically what Unreal Engine does. Probably not too difficult for a virtual tracking studio now as it is
You guys do know the sync e doesnt need that cable to work w bm camera right… it pulls it directly over bluetooth in the same fashion of how the sync e’s talk to each other. no cable needed.
This is a call out to like RED, Arri, Sony and Blackmagic. This NNNEEEDDDSS to happen. I love the idea 💡 📸🎥
agreed!
Couldn't you Do the same with A VR Headset or an FPV Headset?
maybe focuspull but you normally need to see the markings on your remote and also, as she handled the camera while wearing the VP, this wouldn't be possible in VR, only AR let's you interact with your surroundings
4:30 vision pro can at most display 100nits brightnes. Also you view it from so close that it is not enough resolution to not see individual pixels. And you are better off looking at a phone or a real monitor.
thanks ! aweisom !!!
It’s a great use case for using Vision Pro, imagine when developers start making proper tools for film makers on Vision Pro. The only thing I would say is this is definitely mixed reality and not true AR. I think you couldn’t ever get this good with true AR, no matter the form factor. High quality displays in what is basically a mixed reality headset is the only way moving forward and with slimmer design and a broader field of view, it’s exciting to see what’s next.
What about the lag
We found that there wasn't any noticeable lag from camera to headset. There's always a little latency between any lens, sensor, video signal and wireless transmitter, but we felt this was about the same as a normal cinema camera and didn't impact our focus pulling. There are other stability issues with timecode, the app and the weight of the device, but we were surprised latency wasn't an issue!
Nice! Next to put 3d eyes on the camera 😛 together with the sonar distance
Im a 1st ac and this tickles my brain now to find the cash for an vision pro
Comparing the price of a full Preston FIZ kit? This is nothing
@@Eddie32791 oh I do know my brother! I have a WCU-4 and Nucleus out in the rental circuit .I just admired but could not justify paying for the vision pro but this swings it in my favour
That looks pretty fascinating. However, there are still some problems that need to be resolved. As a camera assistant myself, I always have to be able to rely on the equipment, and that doesn't seem to be the case here yet. Relying on third-party apps can really cause problems on set. In addition, streaming via WIFI and HDMI creates a latency problem, and as a 1.AC every millisecond counts. But nonetheless, I find this kind of experiment really enjoyable and hope that it might become usable at some point.
I mean, very interesting concept but i think that there should be a lot more to the apple vision pro than just creating a virtual monitor somewhere on set. If you want more mobility you can rig a 7" directly on the hand unit of the focus kit and walk around with that and I think that would be cheaper (and safer) than having someone with weird goggles on set all day. Also an awful lot of focus puller (maybe more seasoned ones, but everybody should work this way) prefer to use the monitor as a focus checking tool and pull focus by measuring distances, using marks and just seeing distances by eye, it's a bit more difficult and you're basically pulling without seeing the image but in this way you can be always aware of where the camera is in relation with the actors. If someone miss their mark if you are just looking off from the monitor you are not aware of that until you see the picture out of focus and in that case there is nothing you can do, there is an out of focus part in the shot.
I think that if we want rally to implement the Vision Pro on set we should use it to create something that it would be otherwise impossible, maybe combining the lidar scanning, mapping the lenses and the camera so that you can see the frostrum of the lens, the current focus position in AR, the minimum focus distance, maybe you could be able to add marks. Also you could have camera infos floating around without being relegated to a rectangle somewhere.
I don't want to criticise your work and your video, I know that something like I've described doesn't exist yet and the vision pro just came out so everything it's new, but your video made me think a little bit about that
Great idea, but for the price of the Apple Vision Pro, you could buy a 6K Cinema Camera. I wonder which one a small film studio would choose to buy?
Very cool, has a lot of potentials! Can I use Quest 3 for filmmaking as well?
If black magic comes with the apple vision pro app to seamlessly control the video rig, next gen vision pro will sell like hot cakes(if the price is affordable) 😅.
On what camera has this UA-cam video been shot? Honest answer please:)
Hi Peter! The stuff of me in the office is shot on a Panavision DXL2 using PanaSpeed lenses (a 40mm to be exact).
I have a custom LUT developed and use a Creamsource Vortex 8 as my key light and Aputure Infinibars for a backlight and rear wall light.
@@Strada-Tech So it is not shot on an iPhone with special DOF adapters?
In terms of the actual future, I think the Vision Pro is the camera; computational photography simulates cameras and lenses; depth mapping recorded for VFX and of course, the VisionPro viewer of the movie can take over eye-tracking focus from the camera operator. Other spatial camera devices will make positioning and moving the camera possible, so you aren't tied to first person point of view
Let's hope so! Currently, I even cannot use my iPhone 15 pro (USB-C) as a camera monitor... -.-
Great effort but you didn’t realize a major issue with the Vision Pro …latency. Until this gets resolved I highly doubt the Vision Pro will be used to pull focus remotely on pro shoots.
We were surprised, but didn't experience noticeable latency that was more than any normal cinema configuration. Felt very much the same in terms of latency (probably somewhere around 300ms give or take in my estimation).
@@Michael_CioniYou seeing latency vs a professional focus puller in a shallow DOF situation are two different things. Plenty of people who use “low latency” wireless image transistion devices don’t think they see any latency. But it can make or break a shot extremely easily. There is a reason why Teradek is the gold standard in the industry
Latency is the big thing for me in this situation. Using a wifi based system is the worst possible option when trying to fight latency. That combined with the inherant latency and inaccuracy from the nucleus is a recipe for disaster on tight shots.
Indeed. I agree with you. I have several Teradek Bolts. To clarify what I'm referring to is the innate latency between a lens, a sensor, an image processing pipeline, a video output signal and the wireless transmission. Even tho Teradek's are marketed as zero latency, there is latency before they are given a signal. In other words, it takes a little time to get light from a lens to an any monitor, and 200-300ms is fairly common. Only film cameras with optical viewfinders (and video taps) are able to avoid this. I agree with you that poor latency can absolutely blow a shot. But whether they know it or not, the professional community has become accustomed to acceptable latency due to digital camera image processing pipelines. I'm intimately familiar with professional camera systems and monitoring latency and from my experience, this very early iteration of AVP wireless monitoring is on par with the latency of traditional cinema systems. Get your hands on an AVP and try it for yourself and LMK if you agree. Thanks for the comment!
Just make sure to have 15 batteries to last the whole shoot.
No you could easily use portable charging block
4:45 I feel like your failing to consider the chromatic aberration that the vision pro has
I’m not aware of any CA artifacts. I own a set and watch Dolby Vision films that I worked on (so I know the content well on a SONY X300 or 310). I’m not saying there isn’t CA artifacts in the Vision Pro Display, but I’ve not personally seen any. The films I’ve helped color correct look spot-on to me. But since you raised this point, I’m gonna ask around and see what my colorist colleagues say. Thanks for the comment!
@@Michael_Cioni good point. i recently tried a demo and i just had problems with the pass through really. the actual virtual content was so good
awesomeness
Why would you need it
And move quickly?
I can move quicker without this on my head which is destracting my eyesight and range of view plus not to mention my perception and instinct of how someone else near me is going to move
And relatively cheap?
If you can't do it on a high end monitor then get another job
Are they paying you?
Thanks for the comment. To be clear, no one is paying us:) I've been working on high-end cinema for many years and what helped me develop as both a filmmaker and entrepreneur is to remain curious about the intersection of art and technology. Your points about mobility are valid, which is why it's important to remember AVP is extremely new and AR integrations in the production space are almost totally nonexistent. But curiosity, testing, and user feedback will help fuel innovation in this space that will provide solutions and opportunities that may seem foggy today. This has always been the case with new technology in the production space, but eventually, fads become trends and trends become standards. Appreciate the feedback!
Finally a video about the AVP worth watching. Thanks for seeing the possibilities, and enabling capabilities instead of just being another whiner complaining about what the AVP isn't.
Hey! Thanks! Glad this was fun to watch and informative! I always look at things from the perspective of glass-half-full. And with brand new technology, it's all about getting excited about what's possible. Thanks for the comment:)
Not really AR... Apple promotes it as "IMMERSIVE Reality," because the user isn't taken out of the environment. Nitpick, I know... but I was deeply impressed with how you've taken a very basic visionOS release and incorporated it into your workflow. As an actor, it'd also be helpful for us to see the directors vision on a shot, so we'd know exactly how to act for camera (for those directors that use a video village). Overall, very very impressive!!!
The latency on this must be terrible. I personally put a 12.9inch iPad on a chest rig as a back up monitor to a real focus pulling monitor. I send a latency free signal to the focus monitor and then go out from there to an Accsoon cineview SE. it gives me a more mobile monitor and gives a signal to other departments who want a feed for their smart devices. I wear my nucles on my right leg on a leg gun holster.
It surprised us that there was hardly any noticeable latency added by the APower Mirror (app) wireless transmission from the iPhone (running Blackmagic Camera App). Had we had that issue, we would have brought it up here along with other issues we pointed out (timecode $ weight).
Today this setup did work, but it’s not viable this early. That’s why feedback from professionals like yourself to relevant manufacturers is how improvements will be deployed. Thanks for the comment.
Amazing. But that flex on the rod freaked me out. 😬
I really thought this would be pulling in 3D. Having actual 3D spacing would help focus so insanely much. Why else would one really need a Vision pro.
Plot twist: in the future the camera will be the headset.
I may be in the minority here, but I have no idea what “focus pulling” meant and was lost most of the video. Just thought I’d give some feedback to improve your videos in the future.
It's not a video for beginners, it's showing other professionals what they can do. Googling 'focus pulling' takes 5 seconds, way faster than writing your comment.
I like the part where you never talked about how terrible the latency would be using this in the real world. It's just a gimmicky click-bait video with no actual professional application pulling focus on set.
We actually spent a lot of time testing lots of things and the Vision Pro did not exhibit any latency issues that felt different from that of a normal cinema solution. Normal cameras have 200-400ms latency from lens to sensor to processor to wireless transmitter/receiver. Having done that thousands of times, we felt that iPhone to Vision Pro was probably in the 300ms range, give or take (I didn't actually measure it with a hardware tool). But it was interesting that of all that areas for innovation and problem solving, we were really surprised latency wasn't one of them! Thanks for the comment:)
ive always felt like I was born too soon.. this is only more proof
For a bit there I thought this was an April Fool's video!
I had a short career as a cinnamon photographer, went too niche 🤷🏻♂