ReLU is a switch🤔f(x)=x connect, f(x)=0 disconnect. A light switch in your house is binary on off yet connects and disconnects a continuously variable AC voltage signal. The dot product of a number of (possibly switched) dot products is still a dot product. Meaning a ReLU net collapses to a simple matrix when the switch states are known.
Very nice presentation!
Glad you liked it!
ReLU is a switch🤔f(x)=x connect, f(x)=0 disconnect. A light switch in your house is binary on off yet connects and disconnects a continuously variable AC voltage signal. The dot product of a number of (possibly switched) dot products is still a dot product. Meaning a ReLU net collapses to a simple matrix when the switch states are known.
'K?
What is the software you use to create your algorithms.
Thanks for your interest. We use python for all our work. You can find the code referenced in some of our recent papers