Відео

I can try to use gradient descent to solve the traveling salesman problem, but I do not know how well my experiments would work. I also need to think about an objective function for this.

Not that this is necessarily computational more effective...

I am glad that you have found the visualization appealing. Plenty of serious mathematicians have proven many theorems about sets that avoid certain arithmetic progressions. And the combinatorics of arithmetic progressions involves some interesting mathematics including set theory; the infinite version of the an der Waerden theorem has a short proof that uses ultrafilters. And the animation shows many signs that this is an interpretable arithmetic progression (these signs include near symmetry along with self-similar fractal-like structure).

What did Euler did with large sets with no arithmetic progressions?

I am only vaguely acquainted with papers that deal with networks that try to ameliorate the problem of catastrophic forgetting in neural networks. But I barely consider catastrophic forgetting a problem since if one still has the training data, one may continually train the network on that data. The problem of catastrophic forgetting is an artificial problem since one has to remove the training data in order for the problem of catastrophic forgetting to be a legitimate problem. Neural networks need to forget a lot of information in their initializations. I actually believe that neural networks do not go far enough in forgetting; even though the network forgets the specific data points, the cosine similarity between the weight matrices of the network before and after retraining is very high which indicates that the network forgets how to perform tasks, but the network still retains its general structure. I would be interested in constructing networks for which we can recover all the training data from the network without generating any examples that are not in the training data though, and such networks do not forget the training data.

I can try, but the results may be worse than with discrete optimization techniques.

I used the the Plots package in Julia.

Thanks.

The new UI always sucks at first until you forget how much 'better' the old one was.

Thank you for this completely original and totally non-copied joke. It is very funny.

@@josephvanname3377 all in a day's work

As time goes on, it gets more difficult for the algorithm to make improvements.

@@josephvanname3377 Man I love what you're doing, I'm so interested in learning more about you and your way of thinking, if you have any resources recommendations that would be very helpful.

I can do it to sort an array, but we already have N log(N) algorithms for sorting arrays, so I don't think sorting using evolutionary computation or hill climbing would do much good.

@@josephvanname3377it would look cool!

Fruits and vegetables would help though.

I did not bother with optimizing the hill descending algorithm. I just picked random pairs of squares. One simple optimization for this algorithm would be to only compare the loss for the rows and columns that contain the swapped squares, but this code went fast enough that I did not even need to do that.

The solution is to take lots of math courses that build upon each other so that you don't forget. One needs calculus to compute the expected value and variance of a continuous random variable.

The worst part is that if we take the cosine similarity between the weight matrices are different parts of the training, then the cosine similarity will be very high. The network does not change its structure during retraining, but it forgets how to do things. Neural knee twerk.

The network is getting tired because it is getting fat. As the network goes through the data over and over again, it increases the norm of its weight matrices and bias vectors to the point where it has more trouble learning. You can tell that this is the case because the network slows down over time and takes more time to cycle through the data. Regularization and zeroing out nodes would have helped with this.

This is not art. This is just a demonstration of the behavior of neural networks.

​@@josephvanname3377 anything can be art

I make the visualization frame by frame and then I put all the frames together into an mp4 file. How to begin with AI depends on your math/CS background, amount of spare time you have, and level of interest in AI.

As long as you don't forget your cryptocurrency private key or calculus. . . But in reality, humans can generally remember how to perform tasks after learning different things, so catastrophic forgetting does not happen with humans. I have not been playing the piano much recently (though I may have to play more piano to add music to my videos), but I have not forgotten how to play the piano at all nor have I forgotten specific pieces that I have gone over in the past.

QR codes are not as interpretable though. And the QR decomposition of a matrix is not very interpretable either. If you want something looking more like QR codes, you should check out my visualizations about bent functions and Hadamard matrices.

That is a good strategy for more difficult problems. But this is an easier problem as the problem of producing a Latin square is a more difficult problem than this one (I did not want to use too many colors, so I chose an easier problem rather than a problem where one has trouble distinguishing colors). Since this problem is easier, it is not likely that we end up in a local minimum or a region that is hard to escape. And for this problem, there are plenty of moves that change the board but which keep the loss level stationary, so since we can still explore without increasing the loss level.

I appreciate the enthusiasm. Perhaps this content can be played at airports or hotels or in the backgrounds of malls or in the background when people are listening to podcasts.

Here, the waveform only goes through a few periods. It would be better if I used a periodic activation for a longer waveform.

It takes that long to learn.

@@josephvanname3377 I know my friend, I would have reduced the video anyway!

@@matteopiccioni196 Ok. But a lot has happened in those 14 minutes since the network struggles so much.

It recovers the original image by minimizing the loss level by moving in the direction opposite to the gradient. The loss level is the distance of the magnitudes of the discrete Fourier transform. I gave more details in the description.

@@josephvanname3377 owh thanks for the explanation! Now it crystal clear for me😆

@@justman017 You're welcome. I am glad that it is now clear.

Unlike the other network with the conventional activation function, it appears as if this neural network is heating the pipes up to a very high temperature before bending them. But these are my post popular visualizations. I have made other visualizations where the network was doing something really fun, and in those visualizations, people don't want to watch as much.

If the neural network had a periodic activation function it could fit sin perfectly though

There is a big gap between the inability to extrapolate over the entire field of real numbers and the inability to extrapolate a little bit beyond the training interval. And polynomials can only approximate the sine function on a finite interval since an nth degree polynomial has n roots (on the complex plane counting multiplicity).

That is great.