In forget gate, if it is 0 then it is not carried with the cell state because it is being multiplied with previous cell state information, the information is only carried if the output from forget gate sigmoid is 1.
These are great! Thank you both! I am making my first steps in this and if I may ask you questions: it seems like elements of C in LSTM will be near to One when vectors they represent occur lots of times while learning in contrast to GRU where elements of H will be near Zero for those vectors that occur more often then others?
Thank you
In forget gate, if it is 0 then it is not carried with the cell state because it is being multiplied with previous cell state information, the information is only carried if the output from forget gate sigmoid is 1.
These are great! Thank you both! I am making my first steps in this and if I may ask you questions: it seems like elements of C in LSTM will be near to One when vectors they represent occur lots of times while learning in contrast to GRU where elements of H will be near Zero for those vectors that occur more often then others?
HI David I am in the middle of course tsunami ( 200 students) -- I can reply after classes finish!
@@poli369t-sci6 Thanks a lot for your attention!