SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction
Вставка
- Опубліковано 16 вер 2024
- Official presentation video for SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering.
Authors: Antoine Guédon and Vincent Lepetit
You're joking! Oh, man. Brilliant work. Thanks for showing it.
really interested to see how system resource hungry this well be ! super exciting news !! thanks !!
oh wow great 3d mesh from Gausion Splatting
wow... compliments!!
Awesome
Les choses se passent plus vite que prévu.
Love it! Is there/will there be a paper describing this work? Looking forward to it.
Thank you for your nice words! Yes, we actually published the paper on arXiv a few weeks ago, and released the code a few days ago. There is also a dedicated webpage, feel free to check it out! Just look for "sugar antoine guedon" on Google ☺
Let me just humely rais my hand and ask for your Attention so taht i may express my out most respected. Is it just me but did you guys just make 3DGS not just a pretty face but actually usefull. Amazing. I wish i was smart enough to understand how this work but for me its black macic. 🎉
insane.
yeah baby
make them move
gallantmon looking fine af
A man of culture! Indeed 👌
wow
Hi Antoine, what phone did you use?
could you wrap it up in a colab file?
Please make a tutorial
We released the code a few days ago! The GitHub page provides some help to install and run the model, but I will add detailed tutorials in a near future
What camera did you use?
how can i edit and use glaussian splats in blender ??
Hmm, but you end up losing the advantage of Spherical Harmonics, right? This brings it back down to polygonal rendering, if I'm understanding it correctly. (Not to undermine the work as it's probably still going to be far better than photogrammetry in casual captures).
Hi @tmibtruemakers, that's an interesting question!
Actually, we do not lose any advantage of Gaussian Splatting rendering, and we still use Spherical Harmonics. Our hybrid representation is somewhat a new rendering method for polygons, which uses thin 3D Gaussians equipped with Spherical Harmonics to reproduce view-dependency.
Indeed, we first designed an algorithm to extract surfaces (in other words, polygons, you're right) from Gaussian Splatting representations. After that, we propose a refinement strategy that builds a hybrid representation "Mesh + 3D Gaussians on the surface", as we suggest in the video.
This hybrid representation includes 3D Gaussians that are rendered with the Gaussian Splatting rasterizer: Just like vanilla 3D Gaussian Splatting, each Gaussian has a set of Spherical Harmonics that helps to reproduce view dependency in colors.
The cool thing is, if you move or animate the mesh, the Gaussians will follow! Because meshes make sculpting, rigging and segmenting easier, we believe this can help animation and edition in Gaussian Splatting scenes.
@@antoineguedon Thanks for answering, and now that you mention it, it's actually clarified in the video, too. I guess I wasn't paying enough attention. If you've already accounted for it, that makes this technique so robust. Great work! Hope you continue to pioneer such contributions!
The problem with this video is that we can't really see clearly the result, everything is spinning too fast, the frame rate is too low to be clear and then the scene is over in just a few seconds
Does anyone know if it has already been released? If not, does anyone know when it will be released? :o
We released the code a few days ago (just look for "sugar antoine guedon" on Google)! The scripts for composition/animation are still missing right now, as I have some cleaning to do. But for reconstructing scenes and extracting 3D meshes, everything is here!