- 3
- 67 721
Benjamin Joseph Mildenhall
Приєднався 5 жов 2015
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains (10min talk)
NeurIPS 2020 Spotlight. This is the 10 minute talk video accompanying the paper at the virtual NeurIPS conference.
Project Page: bmild.github.io/fourfeat
Paper: arxiv.org/abs/2006.10739
Code: github.com/tancik/fourier-feature-networks
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Matthew Tancik*, Pratul P. Srinivasan*, Ben Mildenhall*, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, Ren Ng
*denotes equal contribution
Project Page: bmild.github.io/fourfeat
Paper: arxiv.org/abs/2006.10739
Code: github.com/tancik/fourier-feature-networks
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Matthew Tancik*, Pratul P. Srinivasan*, Ben Mildenhall*, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, Ren Ng
*denotes equal contribution
Переглядів: 10 247
Відео
![[ECCV 2020] NeRF: Neural Radiance Fields (10 min talk)](http://i.ytimg.com/vi/LRAqeM8EjOo/mqdefault.jpg)
![[ECCV 2020] NeRF: Neural Radiance Fields (10 min talk)](/img/tr.png)
[ECCV 2020] NeRF: Neural Radiance Fields (10 min talk)
Переглядів 21 тис.4 роки тому
Best Paper Honorable Mention at ECCV 2020. This is the full 10 minute talk video accompanying the paper at the virtual ECCV conference. Project Page: www.matthewtancik.com/nerf Paper: arxiv.org/abs/2003.08934 Code: github.com/bmild/nerf Supplemental video with additional comparisons: ua-cam.com/video/JuH79E8rdKc/v-deo.html NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis B...
![[SIGGRAPH 2019] Local Light Field Fusion (technical video)](http://i.ytimg.com/vi/LY6MgDUzS3M/mqdefault.jpg)
[SIGGRAPH 2019] Local Light Field Fusion (technical video)
Переглядів 36 тис.5 років тому
Published at SIGGRAPH 2019. Find paper, code, and more at fyusion.com/LLFF Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines Ben Mildenhall, Pratul P. Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar 2:35 Comparisons to other methods 4:49 Results on 60 real world scenes
the project link is currently unavailable. Could you please share it again? Thank you very much.
what app is being used to capture images in the correct position/ at 1:30
The paper that started a revolution.
really awesome video thank you so much! great work :)
😮
Hi, thank you for the great work. I just wonder what software you used to make this video that could vividly show the iterations, the Fourier features and its Std, frequencies, and reconstruction.
Thank you so much for this wonderful video!
Thank you so much for this wonderful video!
You can just use Discrete Cosine Transform to do it. We have a paper: www.cse.scu.edu/~yliu1/papers/ISCAS2020Yifei.pdf
Stunning
Would it be feasible to somehow incorporate the fourier features in the activation functions? So that the entire model can be made high frequency sensitive instead of just the input
You can just use Discrete Cosine Transform to do it. It's much simpler. No need to use Fourier transform to make it complex. We have a paper: www.cse.scu.edu/~yliu1/papers/ISCAS2020Yifei.pdf You can write 2D-DCT into 1 dimensional representation for activation function. See our another paper: www.cse.scu.edu/~yliu1/papers/ISCAS2021YifeiPei.pdf
However, these transforms can only work on fully-connected neural networks. It gives bad results on CNN.
Such a new concept.. No body really thought about it before them. How genious!!
Could this technique be combined with photogrammetry to be able to represent reflective and plain surfaces as well as people?
Wow
How does it compare to NeRF?
Great video, especially the part with the scale is well explained
Good channel. Looking forward to seeing more from you! By the way, go and search for SMZeus . c o m!! It will help you promote your videos!!
Nice work and good presentation. Just a couple of questions, though. At 5:59 you define the loss as using the function “render” which has never been defined here or on the paper. Can you please clarify where it comes from? Thanks. Also, at 4:46 I understand that C = 𝔼(c), with c vectorial output of the net, following the cumulative opacity, function of σ, output of the network. Is this correct?
I guess, render function means just computing the color of an image at a particular pixel, using the formula, defined in ua-cam.com/video/LRAqeM8EjOo/v-deo.html
A+ in my book.
incredible!!
a solution looking for a home, Rick Deckard would approve
At 9:07, the reflection on the TV screen is also changing accordingly. That's really amazing!
what did you expect, it was present in the input images. for the code, there should be no difference between the reflection and the 2nd room behind the tv
You might not believe, but 6 hours ago, I suddenly decided to read your amazing work! now I am here! Thank you
In about the fourth set of samples there is a dinosaur skull ( on bottom row) that had serious flaws. Please explain why that one failed.
Is there a google colab notebook we can try this out with?
ok but what am i going to do with wobbly 3d images?
Please add vertical capture to fyuse for those of us who are passionate about our captures!
wow
Impressive demonstration of mathematics knowledge. But it is more practical to shoot video for the best result. Isn't it?
the matrix had always been, the only possible goal
Can you capture input data from a video? Would this work with non RGB images? Monochrome or even something like low resolution thermal imaging (320*240)
Empolgante! : )
Wow. And this video itself stands out for its quality too.
Very promising results, great work! Is this method potentially applicable to 360° lightfields?
is it possible to catch instant processes by this method, I mean splash of water or fire??
i assume if you had 25 cameras in a semi-regular grid that all took a photo instantly, it would work
This is so awesome!!!! Also didn't realize you were at Berkeley until I saw that fern, that's VLSB isn't it haha?
Make this into a phone app and become rich.
any chance on someone making an app for this? or even just a more streamlined program to use on desktops?
Really wish there was a Windows port :\
Possibly as the code is open source and has 100+ stars. Keep in mind it’s a remote possibility and all of these tools typically aren’t even shared.
In what subjects is these techniques useful? What do they solve and where will you use them?
@@RivenbladeS They didn't really go over the uses of light fields in the video, but it's really good for VR ua-cam.com/video/OUU2yGHgPQY/v-deo.html
Fyuse app is a limited implementation
It’s soo cool to capture local light field by using just a mobile phone , I tried the google spherical light field before, clearly this has the quality to compere it.
Outstanding! Have there been any explorations into how your method would work with camera arrays--perhaps including video camera arrays?
It works very well on the 5x5 camera array data collected by the authors of Soft3D (ericpenner.github.io/soft3d/). We have not found any public video camera array data to try it on yet, theoretically it should work, though practically the size of the output MPIs might become an issue when reconstructing more than 1-2 seconds worth of data.
@@benmildenhall3169 Incredible work. We might be able to supply genlocked high-resolution video data. Can you work with larger arrays? What image resolution would you prefer? And do you have any technical preferences for camera spacing and orientation?
@@infiniterealities4D Wow the fact that IR sees potential in that means it's really as clean of a technic as I thought.