In less fancy terms, they defined convolution on manifolds in a coordinate-free way. The same idea is used in "natural gradient descent", where you define gradient descent independent of how you define coordinates on the space of statistical models ("statistical manifold") by using its intrinsic distance function ("Fisher metric").
It seems like their way to resolve the ambiguity of the filter orientation (illustrated in Figure 2) is essentially to always rotate the kernel so that the "up" direction is the direction which is the most directed upwards, right? This creates a singularity in the kernel orientation for completely horizontal surfaces, which I think could be problematic in some cases (according to the hairy ball theorem, you will always get this kind singularity if you use a simple, static 2D kernel, since the "up" direction is a non-vanishing continuous vector field). For example, in Figure 2, by using their method, the blue arrow would suddenly flip 180 degrees once you passed the top of the sphere if I understand it correctly. So, if you place one of the MNIST digits on the top or on the bottom of the icosahedron, for which there is no obvious way to orient the digit (there is no clear "up" direction), wouldn't the performance of the network would degrade drastically? It's not clear to me how they project the digits onto the icosahedron, though, so maybe they just project them on the sides and not on the top and the bottom?
Maybe the "up" component would be multiplied by zero if the tangent plane you are working on is horizontal. Just a guess. It seems (but I might be completely wrong) that the method converts directions in each successive tangent plane into their corresponding 3D directions. This would imply that the manifold has to be embedded in 3D (or maybe higher) space for this method to work. The only problem is that what I'm suggesting sounds like it might be too simple to be correct. What do you think?
Three questions: 1. Does information had to be inscribed onto the manifold and this can then be learned or is the geometric shape itself learned/detected in some way? 2. Why the mathematical derivations, when the use-cases then resort back to regular 2D convolutions ultimately? 3. Does the geometry has to be known a-priori in order to define the convolution operations on it?
this video is not my work but there is a correlation between the two that is hard to see. Mad Max: Affine Spline Insights into Deep Learning: ua-cam.com/video/7Q2JhZxNPow/v-deo.html
these kind of videos are really good. To discuss a paper in depth, instead of just loosely telling its premise, is really what we need. Thanks.
In less fancy terms, they defined convolution on manifolds in a coordinate-free way.
The same idea is used in "natural gradient descent", where you define gradient descent independent of how you define coordinates on the space of statistical models ("statistical manifold") by using its intrinsic distance function ("Fisher metric").
Very interesting paper. I did a game of life using a triangular grid with different weights. Great explanations.
Very well done. Thanks, Yannic.
Excellent !!! Thanks.
It seems like their way to resolve the ambiguity of the filter orientation (illustrated in Figure 2) is essentially to always rotate the kernel so that the "up" direction is the direction which is the most directed upwards, right? This creates a singularity in the kernel orientation for completely horizontal surfaces, which I think could be problematic in some cases (according to the hairy ball theorem, you will always get this kind singularity if you use a simple, static 2D kernel, since the "up" direction is a non-vanishing continuous vector field). For example, in Figure 2, by using their method, the blue arrow would suddenly flip 180 degrees once you passed the top of the sphere if I understand it correctly. So, if you place one of the MNIST digits on the top or on the bottom of the icosahedron, for which there is no obvious way to orient the digit (there is no clear "up" direction), wouldn't the performance of the network would degrade drastically? It's not clear to me how they project the digits onto the icosahedron, though, so maybe they just project them on the sides and not on the top and the bottom?
Maybe the "up" component would be multiplied by zero if the tangent plane you are working on is horizontal. Just a guess. It seems (but I might be completely wrong) that the method converts directions in each successive tangent plane into their corresponding 3D directions. This would imply that the manifold has to be embedded in 3D (or maybe higher) space for this method to work. The only problem is that what I'm suggesting sounds like it might be too simple to be correct. What do you think?
Three questions: 1. Does information had to be inscribed onto the manifold and this can then be learned or is the geometric shape itself learned/detected in some way? 2. Why the mathematical derivations, when the use-cases then resort back to regular 2D convolutions ultimately? 3. Does the geometry has to be known a-priori in order to define the convolution operations on it?
very nice explain for the paper
Very interesting video, thanks bro.
Interesting paper and video. It is very similar to my work.
this video is not my work but there is a correlation between the two that is hard to see.
Mad Max: Affine Spline Insights into Deep Learning:
ua-cam.com/video/7Q2JhZxNPow/v-deo.html
thanks for the reference