PyTorch or TensorFlow?
Вставка
- Опубліковано 8 чер 2024
- ❤️ Become The AI Epiphany Patreon ❤️ ► / theaiepiphany
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Should you pick PyTorch or TensorFlow?
You'll learn:
✔️ A brief history of both frameworks
✔️ How they compare in the research community
✔️ How they compare in shipping to production
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ TF vs PT papers: horace.io/pytorch-vs-tensorflow/
✅ Google Trends: trends.google.com/trends/
✅ TF GitHub: github.com/tensorflow/tensorflow
✅ PT GitHub: github.com/pytorch/pytorch
✅ OpenAI blog: openai.com/blog/openai-pytorch/
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
0:00 - Are there any other frameworks?
0:30 - Google Trends (PyTorch vs TensorFlow)
2:27 - Dimension 1: Ease of development & research
4:22 - Data-driven conclusions
5:55 - Dimension 2: Can we ship it?
7:12 - PyTorch is catching up?
7:45 - So what should I use?
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► / theaiepiphany
One-time donation:
www.paypal.com/paypalme/theai...
Much love! ❤️
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► / aleksagordic
Twitter ► / gordic_aleksa
Instagram ► / aiepiphany
Facebook ► / aiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► / discord
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com/
💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► / gordicaleksa
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#pytorch #tensorflow #deeplearning
What's your framework of choice and why?
Write it down in the comments 👇
Note: I tried to make an objective review (data-driven) of both frameworks and their pros and cons.
BUT. I'm obviously somewhat biased as I use PyTorch for all my projects: github.com/gordicaleksa
Although I did write code both in TF 1.x, 2.x as well as in Keras (pure before it became TF's API).
Stay safe and keep (deep) learning!
Tensorflow, keras and tf.data are so straightforward to use.
Please a video on jax and flax for computer vision. Also it would be nice to know your thoughts on this library and its future.
@@quetzal8343 I've heard amazing things about JAX from my friend from DeepMind, they use JAX extensively there. It's even more Pythonic and super similar to numpy - in his words. I still haven't tried it. Not a big fan of jumping around and exploring different frameworks just for the sake of it. I believe in focus on the problem solving part. But I'll give it a look once it gets a bit more popular I'm definitely following it's progress!
MXNET is best i think for complex things
@@johncaling6150 What do you mean by complex? Give me an example of a project or company where people are using it I'd love to know!
@@TheAIEpiphany I don't know any companies that use it for a fact but since it was created by Amazon I think its safe to say that Amazon will use it a lot. By complex, I mean things like gender and age prediction, full imagenet, etc.
Thank you for the thorough explanation. Now I am motivated to go to the gym and learn Pytorch.
@@things_leftunsaid 😂😂👍
I find tensorflow is too abstract and sometimes I can build a model with TF but can't understand how it works; I switched to pytorch recently and I find it more straightforward when building a model.
Yup, pretty much!
Thanks mate, great wrap up. Just about to dip my toe into DL so this was very helpful.
After researching a lot of resources online. I got a perfect tutorial. Really thank you, brother.
CommaAI (self-driving) just switched from Tensorflow to Pytorch recently as well.
I know!
@@TheAIEpiphany Keras is built into TF so its really ease to learn and use TF today. Plus TF supports and works on lots of microcontrollers, and lots of SBC's, this is where PyTorch is left behind in the dust.
thanks so much for this explanation. glad to see the charts and graphs there
Glad that I found your channel. Also glad that I choose PyTorch for research. Awesome work! Keep it up!
Thanks Nikolay! JAX is growing in popularity as well but I'm still betting on PyTorch.
Thank you for the video! Really helpful for me!
You're welcome!
Great video! Just wanna ask something: you mention on the start of the video some frameworks that might be dead, and you list Keras among them. But how come? I mean, Keras runs on TF and I've seen a huge community using it, works really well for deep learning. Also, TF uses Keras in its own documentation, or am I wrong?
Keras is a part of TF now as you said. It doesn't exist as a standalone project anymore that's what I meant. I said it's "dead". 😅
We need more videos like this, thank you.
Very helpful! Thank you so much :)
Great video, well presented. Cheers
Thanks! Glad you liked it!
I was using tensorflow but it is really hard to develop DL model with it. Even the GPU configuration is difficult with it. Now I decided to use pytorch. Thank you so much for this clear explaination
Thank for honest goodness. Am just starting advance data science but was quite sure of which of the two tools to prioritize. But must agree I have found more tutorials using pytorch than TensorFlow especially about new ML models
Glad it was helpful!
Good comparison! Thanks dude!
You're welcome Vlad!
😎 MOST useful video on deciding which! Thx!
PS: Nice touch with the google trends!
wow bro that was awesome, you just addressed all my concerns.
After taking a big decision to learn TF and completing a 4hr course, now, thinking - why the heck I didn't watch this earlier?
Thanks so much for such a clear explanation to make decisions bro!
Still not late hahah, keep crushing it!
I picked up pytorch because it came pre packaged with anaconda3. It is easy enough to learn esp for those who already studied perceptron, mlp, and backpropagation in the past. I installed TF and Keras but have not played with them .... well, just a bit with Keras.
Great! I was waiting for this
Woohoo I am reading minds! Look ma no Neuralink! 🧠😅
neat summary man. thanks.
Good video. Thanks for sharing.
Thanks, glad you liked it!
I have 2.5 years of experience working with tensorflow 1.x and 2.x (both low-high level APIs), I feel tf had become better with time, handling large and difficult datasets is now more practical compared to 1.x, writing layers in keras gives you some good prototyping speed and flexibility, tf.Module let you handle easier custom things and in general tf has many advantages in production level . Since one year I started with pytorch and I am gonna say at least for me it's the "definitive framework". it feels more native when you write stuff and less buggier than tf also you have more clarity about what's going on and why sometimes things just don't work, for research is a MUST and I feel it should be way way! more adopted in industry. One thing they should improve a bit more is the gap between production and model development!
100%! Don't worry about PT not being applied in industry as much as TF atm, once the framework is leading in research it's just a matter of time before it becomes a de facto industry standard.
I think Google is betting more on JAX than on TF if you asked me now.
@@TheAIEpiphany JAX je tu gdje je. Već 13 mjeseci stoji na mjestu.
why is your worldmap on the back flipped
Thank you so much =)
It may be better if you use a lavalier, it is difficult to listen.. Now I checked your latest videos and they are good. What mic do you use(on your latest videos).
And the thing you said about dl4j at the beginning, as a researcher who is used to work in java do you suggest dl4j, does it worth checking?
Yup since then I've been using Rode NTG mic (bought it for 290 euros in Serbia).
I haven't used it extensively myself but it's definitely worth checking out if you're working with Java.
@@TheAIEpiphany Thanks for the advices. A condenser infront of your face can be distractive, It is nice to learn that a shotgun can take this clear recordings.
I've started to work with pytorch for now, following the mainstream.
@@emrek1 Oh definitely, shotgun mic is perfect for my setup.
Good luck! I think you made a good choice you can always switch if a business need makes you do so.
Hey! I'm thinking to learn ML and computer vision. Which is best, as you said in video did pytorch catch-up? Also if implementing this framework to automation industry which would be better?
The field is moving very fast and the latest research is done on pytorch, and if you are a beginner I'd say learn pytorch first to learn the working of the models and get better then you can choose tensorflow if you want to.
can deep learning train on annotated examples. e.g. theres no images .jpg files theres just a csv file with data in it (numbers and categories) can a deep NN train on this or is it nessasary to have the images as well?
Computer vision using neural networks detects shapes in images by dissecting images into pixels. And then associates the shapes found in the portions of the image with the label of the image. If there is no image to detect shapes from then I don't think image-label association would work. In turn, CV wouldn't work.
very informative
Great summary
Thanks!
thank you!!!
"Using backend: pytorch". How can I get rid of such thing when run python code, any help
He could have played Khal Dorogo but has chosen science :P
The comparison will be JAX vs PyTorch soon?
Thanks!
Thanks Man. for such a great video.
You're welcome buddy!
@@TheAIEpiphany Your video just saved my day. I was actually looking for GPU supported version of numpy. Then I came to know about CuPy, Numba and also the fact that TF and PyTorch both can also do the same job. After watching your video, I decided to go with PyTorch. Then I found this video - ua-cam.com/video/p3iYN-2XL8w/v-deo.html . I realized that how easy it is to migrate to PyTorch from NumPy to harness power of GPU. Thanks man. I understand that it took a few hours for you to plan and then create this video. But I can assure you that it saved several hours for several researchers like me.
@@BhanudaySharma506 Glad to hear that man!
Jason momoa does AI 😱
In my free time yeah, I thought people won't notice...you're the first one 🤖
Jax, very flexible, but also fast, faster than PyTorch. And you can still run on TPU. Yes, now support for TPU has been added to PyTorch, but it was recently done, I tried it, it's still raw.
In sense of demo usage - when I get tensorflow model I just need to load it. When I get pytorch model I need to have the model defined in the code. This allows many researchers to "cheat". This research community trend of "kinda open code" but "not really" is disturbing. Pytorch is enabler of this behavior although not the cause, many researchers use its complexity to basically close their code or render it useless for anyone with less than a week to investigate what they've written. As "end user" of those models, where I download and try to run existing models released in papers, it looks like a trend made me suffer. With every year it becomes harder and harder to run the code that comes with papers. I do think pytorch is more complex in many senses for demo runs (less for development).
BTW Keras is not dead - it just became tf2.0
As a newbie what resources do you recommend to learn more about deep learning?
link.medium.com/CFAXvnWiQpb
Check out this one 🙏
It 2022, i need to know if Keras was really dead?
Hey man really nice video! I was wondering if I should switch to PyTorch. I like to implement papers and work on my own projects. But I have only learned TF 2.0, but looking at your video, I was thinking of switching. I feel like I am pretty experienced with TensorFlow and I have implemented many projects but do you think switching now would be viable for someone like me?
Unless you have a reason to switch, like TF slowing down your developement cycle, I don't think you should switch.
But if you have some time to invest yeah why not you should try it out!
I'll try using JAX on one of my future projects.
@@TheAIEpiphany but do you think not using PyTorch will affect my chances in the industry? In job terms.
@@nalinnagar1232 Not in any serious company. The easiest thing to do is to switch the framework you'd need a couple of weeks to a month depending how good you are.
Not knowing maths, and fundamentals of say deep learning would take much more time to catch up than switching between different frameworks. So no worries.
thanks with a BIG smile👍
Good work!
Thanks!
Here are my two cents, I coded an LSTM network using TF (KERAS ) and the exact same network in Pytorch, Not only TF was faster to code, it performed better. No matter what I did Pytorch could not converge ( Performace was erratic ). Yes, I agree Pytorch is good for research and is a lot more explainable. There is a lot of repetitive boilerplate code in Pytorch which I think should be replaced by high-level API like Keras, as part of the framework not an addon like fastai or Pytorch lightning..
I never new AquaMan was so intelligent ??? :) thanks bro great video.. going the pytorch root.
Nice video, I'm new to DL xD.
What winderfulll video!!! 😎😎 Thanks!!! 😁
Glad you liked it ^^
im old not in school , not that i ever learned anything in school , always self educated myself and want to learn AI so i have not made a choice yet, although i see many more names out there besides the ones you mention like open cv , h20ai and rasa to name a few , what do you think of them ? or are they dead in the water? Great video btw thumbs and subb
I agree with most said here but you missed the discussion about the high level APIs. If you program in Pytorch + Modules you are NEVER going to be more productive than TF + Keras, you have to use a framework like PytorchLightning or Skorch but these are not as mature or easy to use as Keras. I think the comparison here is only true for the low-level APIs, the intermediate level APIs (Module / Layer) are pretty much the same (Layers have shape inference tho). I think Keras predates TF since it was originally based on Theano so it has a massive gap with the Pytorch equivalents and its officially supported by the TF team.
That said, I am actually more excited about JAX than TF or PT.
Hi Cristian!
Phew, to be honest if you need high level API you'd usually do 'from torchvision.models import resnet34' 😅 and than add 1 more line and you've adapted it for a new task.
Yep I didn't get into too many details of how Keras relates to Theano, TF to Keras or PyTorch to Torch (Lua) for that matter.
I've heard all the best about JAX from my friends in DeepMind. We'll have to wait and see!
@@TheAIEpiphany Hey!
I am not referring to importing architectures, they are still just Modules, high-level API's like Keras, Pytorch Lightning, Skorch, etc, mostly abstract the training loop, metrics, checkpointing, tensorboard, etc. I think its still pretty standard in PT to just create your own training loop for some reason, it feels like programming in TF < 1.0, it seems Pytorch Lightning is getting traction among the alternatives which is good (but I dont like the API). I still think Keras is much easier to use.
With some collaborators I created this Keras-like framework for Jax called Elegy, been loving the experience, it will sadly be some years before Jax can catch up to TF or PT.
@@CristianGarcia Oh got you now! I get it, dunno I like having some control if you take a look at some of the projects I implemented:
github.com/gordicaleksa
The PT overhead is super small even with the "low level" API.
Awesome I'll check out Elegy!
What map is that in the background?
ah its just mirrored
I like all the great promises of great versatility with hardware usage that TF makes, until you find yourself in recompiling hell, because of dropping support for Legacy hardware, is pytorch any better at that?
And on top of that tf has TPU support but tpu support for pytorch is unstable
To be honest I don't think TPUs will go a long way having in mind that only Google can make optimized software for it.
nice so pytouch is easer to pickup and would be better for projects where I am running a script and doing all my number crunching before hand
If you think about business it is better it is better to have a team that has pytorch expert and service deployment expertise.
, because by the end of the day what matters is the machine learning model and deployment. And they are a separate fields
Great explaination!
PyTorch is easier to learn and craft a prototype (your idea, experiment)
Tensorflow is much better for deployment
the question is when are you going to make videos in dothraki :D nice content man, keep it up
I'm good at competitive programming and maths. I Just learnt python and made 2 projects using opencv and NumPy.
I'm interested in ML and AI , what should I do next ?
I Need some guidance.
Sorry for the slow response but for others reading this comment check out my video on getting started with ML: ua-cam.com/video/7q_OJvQQ7vY/v-deo.html
Or in a blog format: gordicaleksa.medium.com/get-started-with-ai-and-machine-learning-in-3-months-5236d5e0f230
Thank you :)
You're welcome!
Please tell me the laptop specifications for pytorch and tensor flow.
Great content
Thanks!
If I cares about training time, is there a faster one?
Search for some time profiling blogs, PT vs TF, those will be able to help you as that's a very specific question and the answer evolves together with the frameworks/time.
perfect summary & conclusion. My concern with PyTorch is that it's a nicety for Facebook, driven by Yann LeCun - but what if that team leaves and Facebook is no longer interested to support AI research...? Whereas for Google open-sourcing their code & supporting TensorFlow is almost business-critical. So if your start-up/business depends on it I reckon it's a safer bet to go for TensorFlow... - just a thought.
I think that was a bigger concern years ago, but at this point the project carries enough momentum to continue even if that happens.
Facebook is at this point of time as dependent on PyTorch as Google is on TF. So I wouldn't give this too much attention, my 2 cents.
Any updates to this in 2022?
JAX is a new player worth checking out.
It depends what you are looking for as usual. PyTorch is still the most beginner friendly/intuitive.
JAX can be more performant.
Nice... now... how about a "PiTorch-crash-course-for-dummies"... ;-)
a video about why the other platforms you mentioned were discarded will also be interesting...
and a video (with examples) on why PyTorch is gainning momentum over TensorFlow would also be welcome...
PS: ?what's the sh*t with the map??? is it for easy Right-to-Left read??
awesome , very balanced
Thanks!
Jax is raising up now a days
Yep fully aware of it now! I'll keep an eye on it!
@@TheAIEpiphany it would be very helpful if you can make tutorials on that, not now but in future.
@@soumyadrip Sure! Thanks for your feedback man!
@@TheAIEpiphany thanks 🤗
The audio is messed up. I had to turn down the bass on my computer just for this video.
Wow sorry to hear that. It is not as bad on my machine - you know how they say! 😅 Anyways, I've since leveled up and bought a professional Rode NTG mic so new videos should have a much better audio quality! Last 3 videos or so do check them out and tell me if it's better. Thanks for letting me know Alden.
I declare this year 420!
Starting with pytorch :)
Wise choice. 😄 JAX is also worth considering nowadays although I'd still stick with PT.
@@TheAIEpiphany yeah xD
why you have a flipped map bro? xD
Hahha it's not flipped the camera flipped it lol 😅
Proud to say I was like number 1000. Thanks for the information, just starting the journey. Can anyone recommend the first ML course an absolute beginner from the web dev space should take?
1000th subscriber?? If so, much love
julia's flux is pretty good
flux?
Ok, PyTorch it is for me
Still confused
CNTK was fastest for RNNs , Theano had sad death :(
Really CNTK was the fastest for RNNs? How so?
@@TheAIEpiphany Google: CNTK RNN Speed and Enjoy! ( It's sad that they shelved CNTK :( also mainly it died because of no Python 3.X support :/)
nice map, just confirmed earth is reversed square
Tensorflow 1.0 was so hard to understand and debug
pytorch is now so easy to understand and understandable and error can be found easily and solved
You can find Tensorflow 1.0 in museums under 1 meter of dust, probably. Why u remember it ??? Current version comes to 3.0, and it becomes close to pytorch as eager, concise language, especially with Keras inside.
TF FTW
Hahaha I see you are a man of passion!
Could you argument your statement? Why do you use TF and not PT or maybe even JAX?
it feels like a gambling to choose pytorch or tensorflow
Your onscreen text disappears way to fast. You can leave it for 10-15 seconds so people can just read in peace. You are rushing it off for no good reason.
Noted, thanks!
PYTORCH
remind me the game between Pc architecture and the Macintosh back in the late 80's. The Pc, that was crap, was more popular, and sold in huge numbers, but the Mac was much better, stable, fast integrated architecture, used only by some hardcore geeks. The same here, TensorFlow is crap , but because of the sale force of Google, they overwhelm the market with...crap !
Paddlepaddle left the chat room
Pytourch never die !! OK
6:54 Tensorflow Virgins
why not learn both lol
Because. Life.
tensorflow has horrifying documentation. still for me is more elegant then pythorch. new libraries like huggingface use both. everybody should have options like that and make it available for both tensorflow and pytorch users.
Sure having an option is very important! JAX is another player in the town - I'll be learning more about it over the next period.
@@TheAIEpiphany please do share resources asap all I found is documentation
PyTorch or TensorFlow?
Jax!
DeepMinder... hahha. Jokes aside, true. I'm watching out for JAX but if you have some concrete numbers - like how many papers are published in JAX on top-tier conferences that would be a valuable signal!
Non-sens Mxnet dead, wtf are you talking bout, you haven't used gluon yet!
Hahaha I said "almost" dead, in my defense!
Biggest tech companies in the world are using TF and PT, also most of the people from startups I know use some combination of those 2. It's simple - the more people use something the better the docs, the community, the support, the tool.
There will always be people using Haskell, Lisp, Fortran, etc. Similarly for deep learning frameworks.
Is there a reason you are using Gluon and not PT or TF? Does it have something these don't have or is it simply something you got started with and are now "stuck" with it?
@@TheAIEpiphany i started with TF but it breaks on every new version, read the gluon docs and you will change your mind as for pytorch i think its very similar to the gluon API, so no problem there, TF is overrated, gluon is underestated as a former user of TF im well aware..
@@Kaizala1933 Thanks for sharing, yup TF has had that nasty habit of breaking it's API every now and then...
The arrogance of a group of Indian Google engineers and architects really screwed up TF big time.
Hm, what do you mean by that?
@@TheAIEpiphany I think you know what I meant. TFv1 is pretty much junk shit. However, the Indian Googlers who were developing TF did not think so. Eventually there was top down effort to revamp TFv1 to become TFv2. A bunch of principal/senior architects and developers left / got fired.
Pytorch uses GPU more efficiently than Tensorflow
TensorFlow no longer support GPU on windows. This is a major disadvantage to stop using TensorFlow.
My job was using pytorch by cpp in path 6 monthes. The pytorch's CPP document is piece of shit. Luckily, I just quit my job. Be honset, I like pytorch if do let me use it on CPP again.
Pytorch is better
I just want to say, I hate both PyTorch and Tensorflow, those two are the worst framework ever written and ever to exist.
ماعجبني خشمك ابد
Since when Indians start to give themselves Serbian name
Thanks !
You're welcome Yannick!