PlenOctrees for Real-time Rendering of Neural Radiance Fields

Real-time rendering of NeRFs via distillation into PlenOctrees (view-dependent octrees).
Please view in 1080p to see details.
Website: alexyu.net/plenoctrees
arxiv.org/abs/2103.14024

[@chronokoks]

you guys rock! Keep it up. The first time I saw NERF I knew there's a lot to this technique and a lot that can be improved.

[@taesiri]

Impressive (real-time) results.

[@ChipWeinberger]

hugely impressive

[@douglasteixeiradeabreu]

Very good 💯

[@daulet344]

game changer!

[@kwea123]

Why don't you train the plenoctree directly like in nsvf? Is it due to the .refine() method that changes model structure?
Also, it would be great if it's possible to cast shadow on the object by the inserted mesh. Naively tracing rays from the light sources to the sample points and calculating if they intersect with the mesh is very costly (although it works quite nicely, I've tried).

[@alexyu1325]

Good question. So far it seems optimizing a MLP works better than optimizing a discrete grid. NSVF is using NeRF still. People have tried and are still trying alternate representations/architectures for NeRF.

[@dinoscheidt]

Tried it on the latest largest iPad Pro - stuck at 100% Loading.

[@alexyu1325]

Mobile browser is not supported right now due to WebGL compatibility problems.

[@davecardwell3607]

@@alexyu1325 do we know what specifically blocks it? Great work btw.

[@DasAntiNaziBroetchen]

God damn dude, normalize that volume. I can hardly hear you.