PyTorch Tutorial 02 - Tensor Basics
Вставка
- Опубліковано 15 гру 2019
- New Tutorial series about Deep Learning with PyTorch!
⭐ Check out Tabnine, the FREE AI-powered code completion tool I use to help me code faster: www.tabnine.com/?... *
This part covers the basics of Tensors and Tensor operations in PyTorch. Learn also how to convert from numpy data to PyTorch tensors and vice versa!
Part 02: Tensor Basics
📚 Get my FREE NumPy Handbook:
www.python-engineer.com/numpy...
📓 Notebooks available on Patreon:
/ patrickloeber
⭐ Join Our Discord : / discord
If you enjoyed this video, please subscribe to the channel!
Official website:
pytorch.org/
Part 01:
• PyTorch Tutorial 01 - ...
You can find me here:
Website: www.python-engineer.com
Twitter: / patloeber
GitHub: github.com/patrickloeber
#Python #DeepLearning #Pytorch
----------------------------------------------------------------------------------------------------------
* This is a sponsored link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏
this is an awesome tutorial.I am leaving some time-stamps here that can help me later
4:53 Element-wise Addition and in-place addition, multiplication, division
7:09 Slicing Operation
8:57 re-shaping a tensor
10:39 converting NumPy to tensor and vice-versa
14:50 CUDA and how to create tensor on GPU
17:26 Requires grad
By the way, you are one of the most underrated youtubers I have seen, and I believe by the end of this year, you will have at least 10,000 subscribers
Thank you so much 😊
Just noticed this old comment again. We already crossed the 10.000 :) Thanks for your motivational words back then!
@@patloeber great job!
and now there are 42k! :D
@@sergiomurilo758 71.6K subscribers :)
Thank you so much man. I was struggling on my master's thesis, but thanks to your videos I'm finally able to develop the application I needed.
This is my first video of yours channel. I must say you deserve more than million subscribers and thousand of likes and comments. God Bless You! Amen
Thank you very much! Glad you like it :)
Your PyTorch course is simply amazing. This course helped me a lot to understand and start working with this framework. Thanks!
happy to hear that!
This is the best tutorial video for pytorch I have ever seen. Thanks!
Thank you 😊
Thank you sir for such a great toturials your teaching methodology is awesome 💯
thank you :)
I have never seen a better explanation than this. Brother you are awesome !...Please do continue ML videos dont stop keep rocking...Great no words left
Thanks for making
Thank you :) I will continue!
Hey, amazing work! Thank you so much for this :D
Thanks!
Best Class ever!!! You're great.
Thanks for watching :)
wonderful tutorial thanks for sharing
Just Awesome ! Nicely explained pytorch 60 minutes blitz tutorial. Thanks a lot ! Love you man
glad you like it :)
great tutorial, thanks so much
Amazing video! Thanks for your helpful content!
thanks!
Excellent Python Engineer !!! It will be very helpful ;)
Wow, what an excellent video. Thank you!
Glad you liked it!
So helpful!
It is incredible perfect! Thanks a lot
Glad you like it :)
Nice this is gonna help me ton with my projects. Really thanx.
glad to hear that!
Such amazing content..please create more educational videos
Hi .. thanks for the great content. I see that you did not showcase the inplace operation example for subtraction and division, which seems to not produce results as expected. Do you know why ?
Thank you!
Also how did you set up VScode to have your prints be printed in the OUTPUT tab of VScode?
Hey which IDE are you using?
Thanks by the way for the amazing tutorial!
Thanks, there are a lot of tutorials on jupyter but I wanted to learn pytorch in VS code. This would help me a lot coz I don't use Jupyter notebook.
Thank you. Great channel. I will suscribe
Awesome, thank you!
amazing❤
Cool tutorial, thanks a lot. Only thing that confused me, at 12:44, lector mentions that both, torch tensor and numpy.ndarray point to same memory location', yet after I ran id(a) and id(b) it printed different addresses in memory (same thing print False for a is b comparison)...
Hi, thanks for the containt. I woud ask you what is the IDE you are employing.
Thanks for the grea tutorial. May I know how did you run the code directly from the editor?
Hi, there is an extension for VS Code that is called “code runner” . Then you can run it directly from the editor :)
Thanks for this. What is the software you're using here ?
Very nice video! Hope you may make some tutorials about the new released dectectron2 source code~
i will have a look at that
Hi thank you so much. i install vs code and i did all extensions which you used. but code runner doesnt work. when i press ctrl+alt+n output says me "python not found" i change right bottom menu language as a python but it doesnt work. i can see in terminal but ctrl+alt+n doesnt work. can you help me? thank you.
hi , i have a for dimension like this torch.size(16, 512 ,14 14), 512 number of features. how can i export this to the txt file
What environment is this? I installed Pytorch and now I am figuring out which environment to use it in.
Great video series. I am on the M1 mac book pro. I had to run command: pip install torchvision in terminal in order for "import torch" to work. Maybe you covered that but maybe this will help someone.
Thanx 👍👍
my one day is wasted for searching this
what app do you use to code
There is no direct way to provide a dtype while converting from np to torch, right? The statement like
b = torch.from_numpy(a, dtype = np.int32)
is not valid. As I found, you can create additional torch arraya and specify a type:
tensor_a = torch.from_numpy(np_array)
tensor_b = torch.Tensor(np_array)
tensor_c = torch.tensor(np_array, dtype=torch.int32)
Awesome Tutorial series
Just a minor query. How do I multiply tensors, like matrices. Since I thought most pytorch operation would be tensor wide multiplication and additions, and not so much element wise multiplications?
torch.mul(x,y) is matrix multiplication. The element-wise multiplication can be done with torch.addcmul(input, t1, t2) where input is added to the multiplied tensors. You can look it up in the documentation.
What is the difference betweem torch.rand and torch.empty ?
15:17 AMD Radeon is also supports! python-pytorch-rocm package already in pacman repo, installing pretty easy in Arch Linux _(long time ago compiling from AUR took whole day and 100Gb of hard memory)_
ppl worrying what PYTORCH course should they do:
this is exactly what u want. (of course calculus is important)
glad to hear this!
@14:15 I didnt get ur point about being on the gpu in order of referencing the tensors, and I got the result that its matter of the operator not device so if u use a+=1 in any device it will refer to the first tensor but it u use a=a+1 the b tensor wont be the same as a in any device
correct me if Im wrong
btw thanks to ur awesome playlist
I have started learning pytorch from ur channel and I would ask many more questions in folllowing days
with love
yes a=a+1 will create a new reference and so it won't affect b. Sorry i meant this only happens on the CPU, because when you move the tensor to the device with a = a.to(device) you also create a new reference
Thank you very much. Are yuo going to do reinforcement learning?
This is on my list for the future, but it's not a simple topic :)
@@patloeber You are a good teacher. what I fail to understand from lot of people, you cover in simplest way. I wish you continue your NLP tutorial and in the future cover things like GAN's and RL
Thanks!
I know he is using the extension coderunner for the code outpu, but when i try it with code runner, it keep saying python not found.
Wait, I just came here from the first tutorial. How did you go from the console to visual studio? What settings are there to be done? Also, if I want to use pycharm what should I do?
vs code has a built in terminal. same thing with pycharm. just make sure in pycharm and vs code that you use the correct virtual environment
what i did was I added a torch package and it worked fine
Hey, great video buddy, you earned a sub!
There is one thing that is happening in colab (torch version 1.5.0+cu101) which are pretty different than yours:
While converting tensors from numpy to torch (and back), incrementing one variable does not change the value of the other.*
Code:
a = torch.zeros(5)
b = a.numpy()
a = a+1
c = np.ones(5)
d = torch.from_numpy(c)
c = c+1
print(a, b)
print(c, d)
According to what you said, the changes made in 'a' and 'c' should reflect in 'b' and 'd' respectively but that does not happen.
Output:
tensor([1., 1., 1., 1., 1.]) [0. 0. 0. 0. 0.]
[2. 2. 2. 2. 2.] tensor([1., 1., 1., 1., 1.], dtype=torch.float64)
Could you please clarify why this is the case?
Thanks in advance.
Hi. Thanks for subscribing! Be careful. There is a difference between a+= 1, and a = a + 1. the latter assigns the value to a NEW variable a, therefore it does not change the others. I should have pointed this out better I guess. this difference is important not only in PyTorch but in general in python
@@patloeber I certainly didn't know that. Thanks for pointing out the difference.
Just to be sure that I understand correctly, a=a+1 increments the value of 'a', creates a new tensor (or variable location) in the memory and assigns the incremented value to it while a+=1 increments the value and assigns it to the same tensor (or variable location) 'a'. Is that correct?
Correct :)
what is this console and where can we find it?
If I create an empty Tensor, I get the value 0, while you get a very high value in your tensor, why is it like that?
It is similar to how numpy.empty and numpy.ones works. Empty will generate and array/tensor faster as it just references memory to be part of the array but without overwriting the memory (as opposed to numpy.ones/zeros). Therefore you are just left with what was in there before, which can be anything. If you run the .empty generation a bunch of times you should get lot of different results, often 0 but not always.
Thanks for the effort you put in your tutorials! But one thing I don't understand: what is the reason (benefit) of reshaping tensors? I watched many explanations but I still don't get it! Hope you can help me out! Thx
Thanks for watching. There are many reasons why you need reshaping. One is that e.g. different loss functions expect the Y and Y_predicted in a certain shape (see the Softmax tutorial). Another is that the shape of X is dependent on your first layer in your neural network. And when you work with images, a lot of times you flatten your 2D image array to a 1D tensor. Then you also want to up or downsample your images, so you increase or decrease the size. This needs reshaping, too…
@@patloeber Thank you very very much
I noticed that if the Tensor is created using
x = np.random.rand(3,2)
y = torch.tensor(x)
Then even if you are using cpu and you update the value of x, the value of y remains unchanged.
What operation are you using to update x? there is a difference between x += 2, and x = x + 2
what IDE is this ? thanks in advance :)
Thank you for your contents. How can I practive learnings from this video to strengthen my learning?
You jumped the gun a bit moving from the terminal to VS Code without instruction
Plz help
wooooooooooooooooooooooooooooooooooooooooooooow.....thank you
You are welcome!
1:30 Why empty tensor shows some random values at output ??!!
what does "modify in place" mean? I think this si the only obscure point in the video. otherwise, very clear and understandable...
what is that ide?
Would you save this file in the virtualenv that was created in the last video?
I'm not sure what you mean? For all the pytorch code you have to activate the PyTorch environment that we created in the last video
@@patloeber My bad, I forgot how virtual environments work lol
Sir what Python script you’re using to write the codes?
You mean my editor? Visual Studio Code
@@patloeber Yes your editor. Thanks for a quick response.#regardsfromindia
Hi the x.to(device) isnt working for me, how can I fix it?
btw I have cuda availabe and it only uses the device in introduction of variable and wont accept changes
hm strange, this should work. is your code like this?
device = torch.device("cuda")
x = x.to(device)
@@patloeber it was just x.to(device)
@@popamaji you have to assign it to the new x again.
do you really use vs code without autocomplete? :) Pylance addon does it. There is also "Kite" that is AI enchanced code suggest feature
What ide are you using in tutorial?
VS Code
3:30
x = torch.ones(2, 2, dtype=torch.float16)
RuntimeError: _th_fill_ is not implemented for type torch.HalfTensor
maybe in your pytorch version or OS this was not supported. try using torch.float32
@@patloeber Yes, Working
@@jabyerre great!
thanks for the video, btw, just being on Linux or Windows is not enough to have CUDA. you gotta have NVIDIA
yep of course
Fun fact, if one writes b = torch.from_numpy(a.astype(np.int32)), a += 1, print(b, "not modified by reference, because astype was called"), b will not be modified :D
This is how bugs are born :D
Thanks for your vids mate. I succesfully went through the first one but I do not understand how to execute python files because I always get a "no module named 'torch'" error when I try to execute a written file from the cmd. However, I could succesfully execute torch functions when I wrote the code in the cmd. Thanks for any help:)
wow, it only works when I start my program with "python program.py", which I never do since I am used to start my scripts just with "program.py", lol
no you should always execute them with "python your_file.py"
which editor you are using
please reply
VS code. I have a whole tutorial about my editor setup
@@patloeber where is it?
@@misbahanwer3528 www.python-engineer.com/posts/vscode-python-setup/
How to change to run on OUTPUT instead of in Terminal?
you need the Code Runner extension
@@patloeber I downloaded the Code Runner but when i run the code it can't find python for some reason
It is time to subscribe to this channel...One more thing, can yo
u suggest me a book related to PyTorch plz?
Thanks! "Deep Learning With PyTorch" is ok but not required. I can also recomment the tutorials on the official pytorch website
@@patloeber Thank You Dear!
Now I am pronouncing the tensor to tenjor.
Minor mistake: You cannot add x and y. One in the gpu memory and the other one in cpu memory.
both are on the gpu if top condition is right
5:27 y+=x
technically tensors are multilinear maps from the cartesian product of 'n' number of a vector space and 'm' number of the dual of that vector space to that vector space's underlying field. i.e elements of tensor product spaces. :) what we're seeing here is are just tensors represented in a particular basis. when they are represented in a basis they look like lists of numbers or an NxNx....xN array of numbers
I don't get a tensor output. Watched your video on anaconda to get it set up right, but I must have done something wrong. To bad you didn't show how you set it up in between part 1 and 2. Subscribed anyway
hmm what error message do you get? Thanks for subscribing!
@@patloeber I don't get an error I just get "exited with code=0 in 0.059 seconds". Thanks for answering!
@@zibiax this means your code ran successfully! maybe you don't have the print statements?
@@patloeber I do have the print statement, but now it just prints "tensor([0.])". It works as it should in the terminal
got it working. Had to mark all the code to make it print. 1:20 Thank you very much for the help!
my empty tensor is just full of 0's while yours has actual values in it
Your accent is adorable
Also show errors
HOW TO CONVERT A LIST OR ARRAY INTO A PYTORCH TENSOR?????
You can easily do this with torch.tensor(list), or torch.tensor(array)
Can someone tell me what IDE is he using?
Visual Studio Code from Microsoft. Its open source and free.
Yes, I have a tutorial about my VS Code setup
Thanks!
watch at 1.75x speed
Very useful and informative but way too many ads. It disrupts the learning process and defeats its purpose.
sorry about that :(
You speak a bit slow the video rather seems better at 1.5x
tenzoa nhi yarrr tenzor hai
No offens...brothers
me: tf is his voice (tooooooooooorchhhhhhhhhh....)
mind: nah his gay
Man, am I coming here for the ads? The ads even take long time than the main contnent
Okay, the way you pronounce "tensors" is super german
Haha, yes I can't hide my accent ;)
too many advertisements