Photogrammetry workflow for surface scanning with the monopod - Gravel PBR Material
Вставка
- Опубліковано 3 лип 2024
- #photogrammetry #scanning #monopod #pbr #workflow #metashape
In this video I present full photogrammetry workflow for PBR materials with the texture used to carry color information for baking and monopod used to stabilise the camera.
In details I present:
- surface capture with the monopod used for camera stabilisation
- RAW image tweaking and micro shadow removal in Photolab3
- photogrammetry reconstruction in Metashape with the texture used to cary albedo information
- how to build and align a lowpoly model for PBR baking in ZBrush to get clean hightmap without any crossurface gradient.
- texture baking with the Substance Designer baker
- manual seam removal, aerial tweaks and bugfixing in Substance Painter
- how to generate a roughness map and equalise the color in Substance Designer
Material is available for free to everyone on Artstation store:
gbaran.artstation.com/store/m...
For those interested to dig a bit deeper into photogrammetry details feel free to grab my ebook:
gum.co/YanD
I hope someone finds this video useful and informative
If you like it and want to see even more like this one, please subscribe to my channel and leave the thumbs up.
Big thanks who did it already
Cheers!
Grzegorz Baran
I just learned why my textures are so bad lol XD now I know what I don't know, time to learn
Cheers! :) I also have still a lot to learn :) and I guess it never ends
So much useful info, I am really enjoying your work and the workflow seems very solid! I am looking forward to applying this myself these days!! Thank you and keep up the great work!!!
Thanks Teodor, happy to hear that :)
great work!!! Thank you :)
Thanks for watching :), happy to know you enjoyed the video
Excellent, thanks for sharing!
Thanks Dave and big thanks for watching :)
Awesome !
Cheers! :)
Thank you for taking the time to share your knowledge with us!
You are welocme :)
Amaizing video! So lot of information in so small "box". Thanks for you hard work.
You are welcome. Glad you have found it useful :) Cheers!
Awesome workflow. Thanks for this. Much appreciated.
Thanks a lot :)
Excellent and detailed video! Really awesome scenery as well :)
Thanks Barny, just found it would be a shame to not include these shots I took anyway at the end. I really love this spot.. its just freaking, crazy beautiful :D. Cheers!
Amazing work !Thank you .:)
Thanks
Amazing video, thank you for sharing this knowledge!
Thank you Dan, happy to know that you found something interesting in here :). Cheers!
Very best!
Cheers!
incredible watch, so educative. thanks!
Thank you, really grald to hear that. Cheers!
Great video! extremely informative and useful. thank you!!
Thanks :)
great content!
Thanks, doing my best :)
Very useful and precious info. Thank you
You are welcome :)
Thanks man it was great and super useful
Im sooo happy with new things i learned from this video
Super happy to hear this and big thanks for watching.
very nice work!
Thank you Gabriel, appreciated :)
Incredible, so much work goes into textures :o
If you expect quality in large scale, you need to provide quality in a small one first. This is just one of many bricks used to build CG worlds :) Cheers!
Thanks for teaching the all process to create a good and realistic texture.
You are welcome Ricardo :)
Hey man, it is so impressive. Thanks for sharing.
You are very welcome :). I am really happy to know you have found something useful in one of my videos, Cheers!
Really nice, good job
Thank you Stephane :)
Awesome work! Helped me out a lot.
Thanks, glad to hear that :)
nice dude!!! like it please more of that stuff
On it!
30 minutes spent :-) Nice job.
Thank you :-)
amazing
Thank you!
A very interesting thing
Thank you Mircea
❤
Cheers!
Great work and learning video. I was about to write to you that most texts about photogrammetry recommend not to correct for the lens distortion and I see that you incorporated in this video. Other questions I had in mind: 1) What focal length do you recommend? Because the more close you get to the ground the more photos you need in order to cover the area with 80%+ coverage. For example with a 35mm lens and coverage 2.2x2.2m I needed around 600 photos. 2) When do you use the polarizer? Do you use always or only when photographing reflective surfaces? I ask this because with the polarizer I get 2-3 stop less light. 3) Why not fix topology in Zbrush with dynamesh? That is ofcourse if needed. 4) And my last question is answered in the video as well. It was about generating the texture colors and not vertex colors. Thanks for sharing all the great work.
Thanks. Yeah, I have coverered details of geometry correction in my previous videos :).. I used to make the mistake of applying them before. But we learn all the time and everyone makes mistakes. My intention is to help people learn from mine :D
ad1: Focal length depends on sensor size. For me with the cropped sensor Canon80D has I would say 35mm is the best focal length to terrain capture. This way you can cover 2by2 meters with 100-150 images.
ad2: I dont use polariser for standard surface/terrain/environment capture. I would use it when I use the flash light which ovewrites the lighting. I am going to cover this in my future videos when I get one.. so far trying to save some money for it.
Also I would use it for photometric stereo captures.. same rule applies.. of nothing changes I am planning to cover it in incoming videos within a month or two.
Ad3. it would need to build back part as dynamesh closes mesh. Also you would double/tripple the polygon count. FInally the plane covers pure 1:1 aspect radio by default.
Ad4. :D
Happy to know that you found this video useful as it motivates me keep going and put even more effort in next one, so really big thanks Dimitrios for watching and commenting :). Cheers!
@@GrzegorzBaranArt Thanks for the reply. I'm following your workflow in replicating a very similar material. After scanning and exporting all the appropriate FBX files, when I'm trying to bake the textures in Allegorithmic Designer I'm getting some weird artifacts in the height and normal map. There some white spots and lines from one edge to the other... Any ideas what could be the problem? Here is a smaple file imgur.com/a/uovc7B6
Thanks thats a great video, really helpful. btw, looks like south shields in the NE England, is it? recognise the wind turbines and lighthouse.
Thanks :) And yeah, great spot! This is exactly the South Shields :)
Woody Allen approves the soundtrack ;)
:D
Great and detailed workflow that you have created and also good video editing!
I see you also uploaded videos regarding Substance Alchemist material creation. How would you compare Substance Alchemist's multi-image based material creation to your more dedicated material creation workflow? What are the advantages/disadvantages between the two workflows? I would be grateful if you could share your insights :)
If nothing changes I am planning to release a video about this soon. Photometric stereo based materials (multi image based creation) is very accurate. Those based on single image are more or less but still just a guess. Unfortunatelly photometric stereo technique is limited by lighting conditions and light consistency what in result limits capture coverage to basically max 20x20cm. It is a great technique tho as shadows are very reach source of height information.. especially when you use the right one... but as I mentioned.. I am going to cover it in a separate video soon. Cheers!
Thank you very much for this information, Grzegorz. I'm following along your video to make my first PBR Texture, but I have a problem when you import your baked textures into Subs. Painter: how you set up your UV Plane to show the seams? I'm trying to create a plane in Blender that matches the one you have, but when I apply the textures in each channel (in Substance Painter) It doesn't show the seams as yours. I already try to change and modify the plane and the UVs directly in Blender but still, no results. Can you tell me how to set up the plane, please?
Great videos!!! Greetings from México.
I covered details of how I uvmapped this plane and attached as FBX to my ebook about the photogrammetry:
gum.co/YanD
since it was a bit complicated..if you want to keep it simple... just UVmap a plane to fill entire UDIM space 1:1 and you should be fine. Cheers!
I'm curious about texture scale, do you have any recommendations? You ended up using a plane to bake out your textures of a real-world scale of 1.8m. So the tiling texture would have to repeat every ~1.8m. For a fairly homogenous texture like this one it would work well, however even slightly more distinctive texture details would make this tiling very obvious. Do you ever work with larger texture scales (say, 4x4m) to reduce the visibility of this tiling, with appropriate texel density of course, and if so how do you find working with that data (this 2m square was 50m polys, a 4m could easily hit 200mil)?
Hey, bare in mind that usually you dont apply single texture alone and leave it as it is. You should use additional tools to break it. I am talking about tools like:
- propping for environment or for props itself.. geometry details
- texture blending (you can blend two textures using low resolution mask.. derails come from initial material)
- decals - you can bring additional details and author the surface by using decals
What really matters is the texel, there is nothing worse that blurred PBR material which lacks light response.
So o course you can use textures which represents 4x4m.. but I would recommend it only in scenes where you drive a tank and cant see the details from close distance.
So I would say you can create a great and interesting surface even if you capture 1m by 1 m :) if you use more materials then one and blend them together.
Regarding to highpoly density.. it depends on workflow you pick. You can drive highpoly data using temporary texture. This way you dont need 200mln dense poly model. Of course denser model delivers you more details. You dont need 50m polys for 2x2m tho. There is a rule in games that you use 512px to represent 1meter. I described details of it in my ebook.
Finally, it is material artist job to make a texture where tiling isnt obvious but the texture is still interesting. It is a matter of practice and balance and knowledge of how the texture is going to be used. Usually good tiling shouldnt be very obvious when you tile texture 3 times.. I call it a rule of 3. But thats it.
Hope that makes sense :D
What do you think about sony a7r or II cameras.Im mostly planning to go with ground textures and small rock formations for my texture library.Thanks!
The camera brand doesnt really matter. All top brands gives comparable results. Of course there are some minor differences but they are really irrelevant. Whats more important is how well you know the camera you use, so you can always get the most out of it. Of course if you already have a certain camera brand for a while, you probably managed to collect set of lenses for this camera, set of spare batteries, etc. therefore brand switch isnt as easy anymore as your lenses and batteires and anything else camera dedicated becomes useless for you.
I had an opportunity to use Sony A7Ra with GN lens for a while and its a very good camera. Also its very recommented brand by photogrammetry community. For me SONY feels a bit like a bulky calculator with optics and doesnt really suit me :D. But if my all lenses for Canon would work with Sony, I would consider SONY as a brand to pick.. but since they not, my next camera is going to be a Canon again. Not because I love Canon but because I know it very well and have entire collection of crap made for this brand.
So as said, all top brands are fine and you should pick the one which suits you the best and which you can afford. Olympus, Sony, Nikon, Canon.. all are great camera brands to consider and there is no clear winner between each. I would rather focus on features of certain type (matrix resolution, lenses choice, options it has, sensor size, battery life, speed, sealing etc.) then brand itself. Hope that helps :). Cheers!
Hi, great video. I am having a problem transferring texture from map. The output log in designer tells me there is no uvs in the high poly model which is almost always the case. All other maps work fine. Any help would be great.
Thanks Dean. Did you generate the texture in a Metashape properly and exported the model after it was done together with the texture? If yes, there are UVs coordinates, otherwise the texture wouldnt be visible on the mesh. To make sure this is the right model, you can also try to preview texture applied in a Metashape. Last but not least, are you sure you export the right model? Not a decimated one etc. as a high poly for baking? Hope that helps.
A brilliantly informative video! Would you use this same process for generating roughness maps when scanning objects too or just surfaces?
Thanks! Regarding the roughness.. it all depends on the subject as well as quality and type of data captured. But after years I still consider this appoach as a decent one.. at the end whatever works fine is fine :). I guess I should cover subject of roughness much deeper as it makes it much easier to make one when its well understood.. its a bit tricky subject tho and there are no a perfect ways to make one, each of these I know is a kind of workaround :) if you know what I mean. So .. its somewhere on my 'TO DO' list.
@@GrzegorzBaranArt Good to know. I haven't had much experience with Substance Designer but I'll definitely be giving it a try to create my roughness maps. The current method I am using is from JasperD's method of capturing two sets of images; one set parallel polarised and the other cross, and subtracting the latter from the former in photoshop to create a set to build a roughness map from. It's a fairly lengthy process to do this with hundreds of images but it's given me some good results; however, I'm trying to keep it as a back up method rather than a primary one due to the time consumption.
I think the whole photogrammetry community would greatly appreciate and benefit enormously from a Roughness Maps explained/ workflow video from Grzegorz Baran :P
Many thanks for taking the time to share your experience and knowledge!
Wow! 9 hours to process, I have assembled new PC with core i9 9900k, interesting how much it would take with mine PC
I am still on i7 and pretty slow RAM and quite old now.. GF1080 graphics card. Also I dont use any SSD drive to store the project file even as I know it should signifficantly speed up overal reconstruction. But reconstruction speed wasnt my top priority target since I simply left my PC to process everything at night and when I woke up everything was done ready for next step :). I guess your new setup should do it way faster :)
Thank You for the video. I did not know about FBX limitation. Thanks for clearing that up! Also nice trick to use normals[3] channel!
Great video, as always. BTW, what flash are You saving for? I can only recommend fnv R-300. Unless You are after some stronger ones.
Hey, thanks a lot :). Initially I was saving for Godox AR400 but as someone told me recently Godox doesnt support it anymore and they stopped the production. Personally.. there is no exact flashlight type I have to buy and I am open to experiment. I know that it has to be very strong flash light which can hadle a couple hundreds shots in series without much delay between each. It has to be strong enough to overwrite existing outdor lighting so the cross-polarisation works. As I see the FNV R-300 you mentioned looks very promissing and is way cheapper to Godox. Cant find what brightess it has.. to compare AR400 has max 400 LUX in 0.5m range.. I guess it can ovewrite even a sun light if it isnt a middday.
@@GrzegorzBaranArt Here is leaflet specification for FnV i.imgur.com/v3b94lG.png Sorry most important number is blurred. it is 2205lux. And it is not flash, it is flashlight- light stream is constant. Example i.imgur.com/3otxF4e.png
@@RokasKontvainis Thanks a lot. Looks like interesting option to replace AR400.. especially its cheapper. As I see it is 349 grams heavy and can be mounted with the adapter directly on last lens ring by default. I guess any camera movement would twist the lens ring and change the focal length. As a solution I would need to get 'L-Bracket' .. hope it has option to be mounted on a tripod or monopod. Unfortunately due to this sh*ty virus cant find it anywhere in the UK :( so I guess it might be better to wait. Do you know am I going to be able to mount polarisation filter with the adapter and light mounted?
@@GrzegorzBaranArt It can be mounted on last lens ring. There are 3 sizes adapters by default. I had to buy additional adapter rings as my lenses are smaller. It works fine if You dont want play with polarization ring rotation. If You do want to play with polarization rotation- then LBracket is needed. It does have both male/female screws and is perfectly designed to be fitted between tripod and camera. Highly advisable add-on. You are more flexible with LBracket. I also bought milk filter but it is of little use for me yet.
@@RokasKontvainis Thanks :).. looks like I need to wait until the Covid19 is gone then to get one.. if I survive it of course :). Big thanks for your help and stay safe. Cheers!
Do you think a polarizing filter is necessary for all situations? I mean in a day with slight evening sunshine and doing a concrete bricks scan, Also, why do prefer substance designer over marmoset for baking texture maps
Nah, you dont always need a polarisation filter. There are certain cases when it is useful and cases when you can proceed without it.
I prefer Substance Designer baker because Marmoset Toolbag one is constrained by the video memory. So in practice you cant really load any heavy geometry if you dont have enough video RAM. So Marmoset is purely GPU based.
Substance Designer offers you a choice.. for small meshes you can go with the GPU and Video RAM which speeds up the baking process the same way it does in Marmoset Toolbag.. but with larger meshes you can switch to CPU mode and handle much heavier meshes this way.. as these are contrained by your computer RAM.
So for dense, high quality scans Marmost Toolbag baker is basically useless as it even can't load them for baking.
XNormal is another great baker really worth to consider. It handles even more to Substance Designer baker.. but is also much slower.
Hi Grzegorz at the 15:54 and 16:59 minute Once the Low Poli mesh has been correctly aligned and Proyected, how do you get back the Low Poly mesh to his previous flatten state before unwraping it and exporte it?
Not sure.. I dont need its previous state anymore.. I need aligned state since its used only as a baking canvas to project PBR data. If it would be flat it would get big details baked instead of small and would be hard to tile as it would get terrible gradient across. SO when I export it I use it for baking and thats it.
hello my friend. your videos are great, thank you!!! i have a question though. when i want to bake my textures. my normal map turns black and my ao turns white. texture and height work. do you have an idea what this could be?
My first quess was that the aligment between high and low poly model wasnt correct or baking distances were off - usually low poly is rotated in a different direction when exported - but since you said that you get proper height map and color map thats not the case and to be honest, I have no clue what it might be. Also you didnt tell me what baker you use. So I would suggest to play with certain map setting. Not sure what do you mean that the bake runs very quickly.. does it still take time to load the high poly model and just baking process get skipped? If thats the case I would blame being out of baking cage (baking distance) or incorrect projection angle but as said.. it makes no sense if you get height and color. AO might be white and being baked quickly if you didnt set any rays for processing but it has nothing in common with normalmap tho. So I cant really help you more without any details as it can be anything - but highly likely just wrong setting.
@@GrzegorzBaranArt hey, thanks anyway. i found out in the meantime that it must have something to do with the FBX format. with OBJ it works.
Couldn't we skip Substance Painter by using the Make Tile function in Designer? Further more couldn't we skip Metashape and zBrush all together with the scanning workflow in Designer (magazine.substance3d.com/your-smartphone-is-a-material-scanner-vol-ii/)? Granted, we wouldn't have such a detailed low resolution mesh. I'd love to hear your thoughts.
Dont think so. Each tool does certain things better to other tools. Tiling in Substance Designer isnt the best as it is based on simple masks which doesnt follow the surface structure. This is why ArtEngine does the job so well because it reads actual data and try to follow it while blending different parts of the texture together.
I would say that seam removal/tiling tool in Alchemist might be a good alternative as it utilises content awarness algorithm supported by the AI. It still doesnt work very well but its very promissing.
Painter is the top solution so far as it use your experience and mind :) and so didnt see any AI which can do it better. It is the only one solution when it comes to surfaces with strong logic like brick walls etc. which needs way bigger chunks of data to be correctly understood by the AI.. its still piece of cake for human mind :) tho.
So when it comes to details, Substance Designer based solution is fast but costs a lot of quality as brins a lot of blur in transition areas. For generic surfaces like sand surface, pebbles, rock etc. Alchemist is way better solution and its the matter of time when results are comparable to those from the ArtEngine. But regarding to complex surface.. manual tiling in Painter is still the only and the most reliable option.
Hope that makes sense :D. Cheers
@@GrzegorzBaranArt I appreciate the in depth reply. I agree, Alchemist is very promising with their AI seam removal as well as its delighting techniques. Your Painter methods reminded me of how we handled it for textures back in the Photoshop days.
Either way, thank you for the reply, and thank you for the wonderful content you're posting. It's a wealth of information. You've definitely sparked my interest in capitalizing on photogrammetry.
Hi !
I just started a few weeks ago the photogrammetry by my self first i tryed the meshroom it was.... ok... ?! But now i started useing metashape and right now i only have a xiaomi note 7 with 48 mp camera but.. just in softwarewise u know.. they fake it.The models are blubish sadly.If i would buy a used Nikon D5300 the 3D scans would be much, much better than with this 12 MP camera which can fake the 48 mp ? Or what camera would you recommend that i can do everything with it like you ! For commercial project quality expl: indoor, outdoor, maybe drone mountable to start with.
18Mpx is considered as a minimum for photogrammetry reconstruction. The lens is more important to the camera though as the lens is the actual eye. I would say that the camera is secondary in here. Any DSLR or mirrorless camera which shots RAW and has manual mode with the matrix larger to 18MPx will be fine. I would say that currently the 24-26Mpx is a sweet spot.
So if you want cheaper equipment I would suggest to get a real even second hand camera. All well established brands like Canon, Nikon, Sony etc. are the same great choice. As I said, the lens is more important.. if you want to save I would also consider a second hand market for the lens. Just make sure you get a good one from a good seller without any issues etc.
@@GrzegorzBaranArt Big Big thanks :)
@@ervinpolyak5779 no probs, anytime :)
Who manufactures your rulers?
I use two different rulers.. I was playing with really different types and have found that type of rulers the most useful so far for the job. One is a plastic one and the second one is made with wood. I think both were made by Stanley ( just found details of this one: Stanley 0-35-229 2m Plastic Folding Rule .. I lost details of the wooden one). A plastic one has more visible graduation to the wooden one. If I have to pick the 3rd one I would probably get the plastic one again. Hope that helps :). Cheers!
Why did not use Artomatix?
Its a really good question... because I usually play with different tools and different ways to do things. Also I value to be independent. I dont want to be too dependent on a certain, single tool.. especially if it is not free and I never know if I am going to have access to it next month .. like a Reality Capture for example.
This is the reason I appreciate Metashape.. since when you have it. you own it forever.
Regarding to what I cover.. I always say that there is no the only one and the best tool or way to go... but to have a real, conscious choice you need to know them all.
I covered artomatix well enough in a few previous videos already and hope to go back to it in a future to present new features which I havent yet but I guess that it would benefit everyone more if I cover more tools.. like different bakers (XNormal, Knald).. different 3D tools like Blender or maybe 3DSMax to build lowpoly, different techniques like photometric stereo capture etc. so everyone has a choice. Hope that makes sense :). Cheers!
эплаудс камерад фенк ю фор зе бест туториал ин зе ворлд +реп бро
Спасибо приятель :)
plz scan some 3d asset
I already did in here where I scanned afull size sculpture using a drone:
ua-cam.com/video/k4btbj6IGhw/v-deo.html
and in here where I scanned a large cliff
ua-cam.com/video/_WJLjOrzNsU/v-deo.html
I am also planning to create a video where I scan small props with the DSLR camera and full turntable setup soon so stay tuned :)
Cheers!
40min to align images, 9 hours to calculate(!), 15min to texture build, x hours to export texture.
Sure, you used a lot of images for a lot of data, but the calculating speed Metashape is working on them is way too slow. I'd rather use another software for this step to cut down the waiting time and get the same result.
Oh.. dont get me wrong.. I can optimise the reconstruction times by playing with the setting .. but it would be a subject for another video. Images can be aligned usually in about 10 minutes, reconstruction can take about 3 hours and still make sense.. etc. But since I wasnt in a hurry and I wanted to be sure I get the higherst possible quality from data delivered .. I pushed all settings as high as I only could. Usually I batch entire process.. so when I run it when I go sleep and when I wake up in the morning everything is over and ready to use.
There is no photogrammetry miracle and different software usually will do it faster only when will process it in lower quality, lower accuracy etc. I guess I can do entire reconstruction with low setting in 10 minutes in Metashape :). Cheers