RNN From Scratch In Python

[@vikasparuchuri]

Hi everyone! You can find the lesson for this video here - github.com/VikParuchuri/zero_to_gpt/blob/master/explanations/rnn.ipynb . And the full list of lessons in this series is here - github.com/VikParuchuri/zero_to_gpt .

[@MohammadKhan-b6p]

Amazing. Every tutorial I've seen of RNNs is just an implementation of the RNN in pytorch or tensorflow with a quick and vague picture of a rolled and unrolled diagram (and this includes paid courses). This is the first video I've seen where I understand how the RNN could potentially process the incoming hidden layer data from the previous iteration.

[@gk4539]

Just a note for any subsequent videos, if you were pointing on the screen it was not visible in the video, and it would be helpful if we knew where you were pointing to!

[@usernameispassword4023]

Shouldn't it be 1 - hiddens**2 for the tanh derivative?

[@kadenabet]

I found this insightgul but very irritated with python nd its eccentricities. For instance int he implementation section, what is params doing there? It looks like a completely useless variable. Should you not update the layers?

[@vubanchowdhury2204]

For multi-layer RNNs, isn't the output from the first layer supposed to be the input to the second layer and so on? From what I understand, the code is written in a way that multiple layers of RNNs will all take the same input sequence (from the original data) and not the output from the previous layer. Could you please elaborate on this?

[@Dataquestio]

Yeah, you're right - I was using single-layer RNNs in this video, so I didn't consider the multiple layer case closely. You would just need to adjust this loop to take in the previous layer input instead of x:
for j in range(x.shape[0]):
input_x = x[j,:][np.newaxis,:] @ i_weight
hidden_x = input_x + hidden[max(j-1,0),:][np.newaxis,:] @ h_weight + h_bias

[@vubanchowdhury2204]

@@Dataquestio Thanks for clarifying!

[@CarlosPfister]

Thanks for continuously offering up free content, even to non students

[@wernerpjjacob6499]

Very good didatic, very good man! I can only thank you

[@goodmusic284]

Thank you so much. This is by far the best explanation of RNNs I have seen.

[@Dataquestio]

Thanks a lot! I'm planning to release some more deep learning vids soon :) -Vik

[@nicolasndour9851]

Thanks for this presentation!
Can i have a clear explainnation about the dimensions of i_weight,h_weight and o_weight?
thanks for advance

[@JennieNewt]

Thanks for your high quality tutorial.

[@minkijung3]

Thank you for this amazing tutorial. I learned a lot about RNN🙏🏻

[@phamquang5535]

thanks, literal life saver

[@AurL_69]

thank you !!!!!

[@BagusSulistyo]

thanks this is awesome 🤟

[@wernerpjjacob6499]

[@anfedoro]

which tool do you use to draw such fancy diagrams ? 😀.

[@Dataquestio]

I use a tool called Excalidraw! Highly recommend it.

[@anfedoro]

@@Dataquestio Thanks.. I have installed Excalidraw extension in my vscode and drawing right there with no requirement to use online web tool.

[@jonathanlowe2552]

Can you please indicate where the csv file is found?

[@Dataquestio]

It's in the code I linked to - github.com/VikParuchuri/zero_to_gpt/blob/master/explanations/rnn.ipynb . If you check in the data folder (same directory it is opened from in the notebook), you'll find it - github.com/VikParuchuri/zero_to_gpt/tree/master/data .

[@jonathanlowe2552]

@@Dataquestio Thank you!

[@waisyousofi9139]

Thanks ,
Where can I get its next video I mean where is the testing step where we can provide our input data.