Photogrammetry - Creating Roughness Maps
Вставка
- Опубліковано 30 вер 2024
- In this short video I show my current process on how I create my roughness maps using a very basic Cross Polarization setup.
Cross Polarization Setup: • Cross Polarization Tut...
Void Scanning method: • Scanning in the Void -...
Turntable Setup: • Photogrammetry Setup f...
Main gear used in this video:
- Sony A7RII with Canon 50mm 1.8
- F&V R-300
- Circular Polarising Filter
- Linear Polarization Sheet
- Manual Turntable
- Reflector (Black side)
Hey, your quality is amazing! Im also here from the reddit threat but commenting to boost your visibility on the allmighty algorythm.
Threat is real
Super useful stuff, thanks mate!
Amazing work and fantastic video. Even though it doesn’t go in to detail it really helps some of us who don’t know all the pieces of the puzzle and how to make our work way more efficient. Thanks
thanks! yea i had the same issue with trying to figure things out and bring it together so i thought sharing a vid like this with the community would be helpful.
Good evening, what aperture, speed and iso do you use for the photos?
with that constant light that i used in this video i used a shutter of multiple seconds. aperture usually around f11 to f18. iso as low as possible. 200/400 iso is usually fine tho, depending on your camera.
AMAZING! ive been looking for this exact video for allmost a year!
nice but you need to have the process of taking photos fully automated otherwise it will take too much time .. I cant imagine manually rotating the CPL filter for like 300 photos
yess very true, i now have this setup completely automated and it works absolutely great!
Awesome job! Thanks so much for sharing! I’m from the Reddit thread.
I would think the reflections you would get would be view dependent from this method. Is it because you subtract? Do you rotate the subject as well or do you leave it on the base?
yes this is on a turn table, so full 360 all around the model.
why do you need both polarized and non polarized image of the same angle?
to exact only the specular info
Hi! thank you it was very helpful, at first i also as you used action and photoshop, and tries to write script for automation. But then realise the it is much easier to use After Effect for substracting one sequence from another in few clicks)
it would be even faster to do it after you generate the 2 textures. 1 diff 1 parallel and then extract the single final texture. but i've found it gives lesss accurate results.
Nikita Kulchitskiy Great tip. Tho I cant figure out how you do this? would you be open to posting your steps in getting this set up in AE? Thanks
Can we do cross polarization with multiple camera setup?
yeah definitely
Do you think this also works properly with a ring flash instead of a ring light?
even better, i currently use the ar400 for this and its great
Have you tried scanning dielectric and non-dielectric metals with cross polarised workflow?
i scanned a wrench once, www.reddit.com/r/photogrammetry/s/1fMIf25qZm
@@jasperd1674 thanks, I have scanned a few metal objects, but struggling with bronze and gold.
so is there any new real roughness map generation workflow for photogrammetry? this tutorial is neat but it's not really a roughness map technically, just an approximation
thanks
so you need to put and then take off polarize filter from camera and ring light each shot? to make 2 image set?
nope, the filter on the light always stays on. and the filter on the lens is just rotated 90 degree for every. so you rotate the filter until you dont see any reflections (cross-polarized) and then rotated until you have the most reflections (parallel polarized). i just put markings on the lens to make it easier. hope this helps!
I was just starting to consider making maps for my models to make them easier to sell. Very good highlights. I would love to see a more step by step video series of this.
thanks! im still working on improving this workflow, to get more accurate results. testing with different light setups, multiple lights, softboxes etc. sadly i dont have a lot of time right now to make an in-depth tutorial. would you be interested in another short tutorial like this but for a different photogrammetry topic?
Hello, how do you photograph long objects? I ran into a problem when photographing long objects they do not fall into the depth of field. Thank you!
not in particular. but i know about the issue. the way to work around this problem is to use a method called “focus stacking”. you take 3 or more images with different focus points, combine them together to get your object fully infocus. sadly this method takes quite a bit of time. also, you want scan long objects parallel to the camera. watch this video from minute 39:30 ua-cam.com/video/REA3XNgUMJg/v-deo.html
@@jasperd1674 Thanks for the reply, I will try this method.
what is the name of the app/program you used?
i used metashape, but this method is not bound to this specific program. you can use other photogrammetry software like for example reality capture.
Thank you, sir! So happy that I've found this video! Keep going!
This content is gold.
thanks! :)
Is it possible to ombine diffused light (e.g. in a lightbox) with cross polarization or better to use direct polarized lighting as you did in the video in order to cut off reflections for photogrammetry of jewelery items?
for the reflections it doesn’t matter if you use direct light or a softbox, because the reflections will be gone anyways. that being said, direct light is brighter then a softbox. only scenario to use a softbox is if you can see the shadows from the object. then it might be useful to get the softbox up. but you’ll need a lot of polarization sheets to cover a big softbox.
I wonder on how you would go about it if the photo session is already over? How would you create roughness etc. without any polarisation sources, only "albedo" pictures?
turn the albedo into a black and white image and adjust the value’s. you could do this in photoshop or any other photo editing app. or even better, in a 3d app. this way you can see in real time how the changes are taking effect.
@@jasperd1674 Wow, why wasnt I thinking about that... I will definitely try that, thank you!
haha goodluck!
Good work man. Quality of the video is also very nice
When taking photo’s, did you do a set of cross polarised and then a set of non-polarised or (this is what i think you did) 1 photo of polarised and then turn the filters off the next for each “pair” of photos. I’m just wondering as the “pairs” would need to be exactly matched and it looks like you have a manual turntable set up? Obviously that would make the actually photoshoot take a fair bit of time and you would need to be very careful not to knock the camera or lights every time… Do you have any tips beyond being extra extra careful :D
I wanted to ask the same question :)
in this video i indeed rotated the filter after each photo, because of the manual turn table. i upgraded to a genie mini turntable that can redo the exact same positions each set. so now im doing first a set cross, then rotate the filter on the flash, then do the same set but parallel. this why you have less imperfections (camera is more stationary). so id recommend just getting a genie mini because it fully automates each set. you’d only have to rotate the filter and object.
Thanks, I did end up getting a genie to solve this. Although it wasn't so bad with the manual turntable as I thought. Results were great, got 94.5% for the practical side of my masters module. Thanks for the video!
I think an important thing to remember is that there is no 100% accurate in CGI. PBR has some scientific basis which is great, and what's more is that it has been adopted by so many rendering engines, but its still a fake. This is a wonderful technique that I never thought of. I have been just editing my diffuse texture into a roughness map by doing some careful histogram selections.
very true! multiple ways to do it indeed, this one is atleast somewhat based on reality but its not really close to actual reality. end of the day: if it looks good, its good.
Here from Reddit thread too, re watching again to make sure i understood!! thanks for the tut
Wouldn't the specular map produced be biased towards the concentrated intensity of the ring flash? Even though these fruits aren't very glossy, you can still clearly see a hot spot from the direction of the ring flash, and if it were more glossy like a mirror you'd of course see a clear image of the ring flash in the parallel polarised pass. Wouldn't it be more accurate to use diffused uniform light completely surrounding the subject?
yes, very good question. haven’t heard that one yet but its kind of an important one.
so even a diffused light source will create a hotspot in this particular case because of the round object. the light bounces into a whole different direction, and thus not reflecting back into the camera. the same happens with a big soft light source (i’ve tried).
but yes you are right it creates hotspots and thats annoying. thats why you absolutely need enough coverage around the whole object. this way you should have a hotspot on every single angle and thus making it “even” again, if you know what i mean. this might take quite some photos tho but the quality of your model wil also increase.
in this particular video i didn’t have my coverage down perfectly and as you can see towards the end when i reproject the specular images back onto the model and you kinda see those hotspots in a pattern around the mango, (not good).
used only 48 images to build the model in that video, so 3 rows of 16 images. also because back then i did everything manually.
a month later i bought an automatic turntable which speeds things up by a lot if you’re doing 2 passes (cross en parallel polarized).
now im doing like 7 rows of 12 images for example, so 84 images. this way the all round coverage is more even and spread out.
hoped this gave you some insight :)
@@jasperd1674 I appreciate the response very interesting. The hotspots you'd receive from completely uniform coverage I'd imagine if anything would just be a faithful representation of the degree of reflectivity of the subject from that particular camera angle, because all those incident angles happen to be contributing to those hot spots, in a way they're like a universal average specular reflectivity. Where as a smaller light source from a single direction like a ring flash only represents reflectivity of the subject from a particular angle. I'm not exactly sure how the specular UV map is stitched together, if it's blending pixels from multiple angles or just using the first pixels shown of that point in the point cloud, in any case I'd imagine the contribution of other incident directions would be lost. I also found it's impossible for parabolic shaped objects like bowls to reflect light from a ring flash around the camera lens back into the camera around the edges of such an object, only a source surrounding the bowl could, so concave objects seem to require it.
my english is not the best so i tried to draw something in paint real quick just to clarify a bit why a softbox wont make a lot of sense for a glossy and round object like a mango/apple. imgur.com/a/0vHBqpT
only in the last instance would you actually get light reflect back into the camera and that could counter the initial hotspots because then the sides would also have much reflection, just as in the center. but that would create problems of itself, cross polarization wouldn’t work in that case for example.
as for the UV stichting, so the pg software kinda ignores the hotspots and just averages everything out, which is not really accurate. the high intensity mode for texturing in metashape is more accurate because it doest blend/averages the images. those hotspots on the texture map would then be the same hotspots as in the original images. so thats more accurate but again this only works with photo coverage from all angles.
@@jasperd1674 Interesting, didn't consider this. The last instance is certainly what I had in mind, I was under the impression that those more extreme angles of reflection from say aside the edges of the fruit or even slightly behind it would still be a faithful representation of the specular reflectivity for those edges, since if we were to use that for the specular map, and shine a virtual light from that same angle, we'd see that same degree of reflectivity from that camera angle. Meaning, the burden of truth should be, if we captured the specular map from diffuse surround lighting, then in blender created that same lighting setup, and placed a camera from the same angle, would we see the same reflections? That test should confirm the assumptions.
Something that is significant about your point though is that if the light were to surround the subject to the point where the light it self could directly pass into the camera lens without even hitting the subject, then that would of course cause unwanted blooming or flaring. Maybe these edge cases are a product of a kind of shimmer, where the incident light is only just grazing the surface, contributing to a higher specular reflectivity. If that’s true, you’re right to think that would bias the measurement. Sounds like we’re referring to the fresnel effect
This is I have been looking for, for eternity. Thanks a lot !
Beautiful work, JasperD. I'm working with a similar approach, would love to connect offline to compare notes. How best to reach you?
offline? well haha, I live in eu :)
@@jasperd1674 I simply questioned going deeper in the comments here, not clear how welcome or not my questions are, but thanks if you're open to hearing...
Good video! Short and simple, thanks👍
that's cool !! i want to make lot of fruits and vegetables like a library for whene i need them and this showed me a good way to start
thanks! yes this is a definitely a good starting i’d say, but there’s still room for improvement. i have already improved on this workflow but due to time constraints i haven’t had the chance to make a new video. goodluck with scanning!
@@jasperd1674 waiting for some new content from u man keep it up
Nice one! :) almost done building my setup. Already so much fun
DUDE. ...THE CUCUMBER. It all looks like REAL LIFE. I freaking love this.
It would be an nice addition if you did another version but for roughness maps of planar surfaces (wood floors, tilled walls...).
It would work right? By taking photos of the surface from different angles with and without polarizer?
thanks dude! planar scanning is definitely something im interested in but i dont have the equipment for it (if you want to scan outdoors). you’ll need a very bright ring flash light to cancel out the sunlight. (or you could try scanning at night. indoors tho is definitely possible using prettymuch the same method. I might look into that more and make a short tutorial like this one :) but don’t expect it anytime soon.
Just a minor, unimportant point, but it's a Zucchini, not a cucumber. I agree it looks great though.
Impressive ! Thanks.
Hi Jasper! This is amazing. Noob question here: I've read somewhere that it isn't recommended to rotate the model but rather you should rotate the camera around it, since the feature points recognition considers the background as well. But clrearly from your video you use a turntable and rotate only the object itself. Is it thanks to your dark uniform background that doesn't have any clear edges? If you had some poster hanged up in the background would that ruin the SFM part?
Thanks in advance!
Yes, thats correct. The background is featureless so it just gets ignored by the photogrammetry software. A must if you want to scan things quickly and you wont have to do any masking.
That is a crazy idea.
That's genius!
Thank you so much for this dude. Going to use this for an object archive project :^)
Amazing!!
amazing video...
good job
Genius
Nailed that.. Just need some solution to shot polarised photos ftom hand.. Great work flow..
dont think thats gonna work unless you have 2 same cameras next to eachother. one cross, one parallel. and even then it might not properly work.
The photoshop actions are nice but not very efficient when working with many images. Any idea how to turn that into a script where you for example pick the folder with your cross polarized images and then the folder with your parallel polarized images for batch processing. Awesome video btw!
thank you so much! yes i agree its not the most efficient way to do it. what you could do that makes this process a lot faster is do the extraction in photoshop after you have created the cross polarizated and also the parallel polarized texture inside the photogrammetry software. then bring only those 2 maps into photoshop and you’ll only have to extract once. but i have found this to give a little less accurate results.
brilliant
Would you recommend the F&V R-300?
its quite a weak light, not a lot of light ouput but with a tripod and slow shutterspeed it works great. if your looking for a really good ring flash id recommend something like the ar400, but its quite expensive. here;s a video that compares the 2 lights ua-cam.com/video/REA3XNgUMJg/v-deo.html im currently looking into working with a softbox tho, to get a better, more accurate specular map result. because with a small light like this you get a pretty distinct hotspot on your object.
@@jasperd1674 Nice, thanks for the info. I've considered the ar400 but it's quite a steep investment for someone getting into photogrammetry. Would you be using multipe softboxes covered in film or just one?
@@wojitechintros im going with 1 softbox for now since a big polarization sheet can get quite expensive aswell. and if it doesn’t really work i’ll just have to use 2 softboxes i guess.
Liked and Subbed dude, please make more videos. Great content.
thanks! would you be interested in any particular subjects? indoor scanning, outdoor scanning, cleanup/retopology?
Amazing, looking forward to your next videos!
i’m planning on releasing multiple videos this summer! also, do you have anything specific you might be interested in seeing?
Great video tutorial, nice and short very digestible. Keep up the good work!
thank you so much!
Great video, looking forward to see more!
Don't give up on youtube, keep up the good work!
thank you so much! i definitely want to keep uploading videos but don’t expect anything soon yet. but i wont give up :)
posted on my bday lol mustve been for me
hehehe, definitely!
nice short video! i think the most shocking thing for me is the fact metashape can bake normals directly in the software?! are using the pro version?
thanks! yes pro version but i think the normal version has that feature as well.
@@jasperd1674 cool, thanks, will check that, never noticed it. Now to the more pressing matter, why don't you upload more videos? ;)
haha yeah indeed the more pressing matter here. i started full time studying again, (something not so related to photogrammetry). and some of my photogrammetry equipment died. but now with the summer break im working on something new and i might make a short video on it in the next couple of weeks. :)
Think it'd be fine if I subtracted the cross polarized map exported from Reality Capture (or any other photogrammetry program) from the parallel polarized map, instead of doing it to all the images before they are put into Reality Capture? I'm asking because I'm considering doing this for scanning larger outdoor objects that won't be on a turntable (the camera will be handheld), and will not have images aligned 1-1 for perfect blending in Photoshop. Also this video is gold and I hope you do more short tutorials, thx!
you can only use this method outdoors if your flash light is bright enough to override the sunlight. if you’re gonna shoot handheld and your parallel polarized and cross polarized images wont be aligned perfectly you wont get an accurate roughness map. What you could do is take only a couple images on tripod so the images do align and and reproject that onto the mesh afterwards.
@@jasperd1674 Alright, thanks!
Very nice man. Always good to see new approaches.
thanks!
What a great video hope to try it soon
thanks!
This video is great! Thank you for your effort to make and share it!
Would you recommend to change the camera from a Sony A5100 to a Sony A7 as yours to improve the quality or just invest on a better lense and a ring light is enough?
an A5100 will be perfectly fine for this type of shooting. actually, even better. the smaller sensor results in less depth of field and thus a more sharp images, its also more cropped in so your object can fill the frame easier. (i had my sony a7 on cropped mode for the smaller avocado). so yes, spend the money on quality ring light and lens.
@@jasperd1674 Cool, thanks for the answer!
Amazing work and fantastic video. I really love your explanation, but I would love to have a tutorial on how you 3d scan in the program because I can't get it to work, and it doesn't work to merge or align the chunks from the images in Metashape and I don't find a solution. I'm getting so frustrated because I want to make a fully 3d model, but I can't find a solution or a fully 3d scanning tutorial. Please help me.
thanks! may i ask, why are you trying to merge 2 chunks?
@@jasperd1674 that is the part I don't know. How to get a clean and complete 3d mesh from various pictures on an object and then in the program make it as one. Because of different technics I have seen, they take pictures on one side then flip it upside down and somehow merges the two together, but I can't find a good explanation on how to do it complete to make one solid floating 3d mesh. I have a hard time explaining, do you understand what I mean?
@@twistedfantasy3131 Hi, the way I do it is as follows. I shoot a round around my object in its upright position. Then I turn it upside down and make another set of photos. Sometimes I even make another round on its side. In Metashape you can then import them as different chunks. For each chunk I place at least 3 different markers on the photos and make sure the are the 3 exact same spots in the different chunks and the naming is the same. Then you can align the chunks marker based in metashape and eventually merge them. That results in 1 chunk compiled with three different positions.
so cool
thanks!
Oh this is really cool
:)
@@jasperd1674 Hi, I found an easier method to do the photoshop thing you do that involves making actions, which is very time consuming. You can just load both the images set in blender compositor as a image sequence and then do the subtract and desaturation using the nodes. Then render out the composite. You get all the specular passes in just one go. 😊
thats smart! gonna give that a try.
@@jasperd1674 You would need to do a bit of changing in the color space to get the desired result before rendering out the animation as single images. It turned out amazing for me. I also tested it out with a mango.😅