For a 2nd year bachelor student like me, who is worried whether he will be able to read these paper with high lvl maths into them, you gave a seriously outstanding breakdown on how to not get scared by papers and tackle them. Thank you for that. Loving this channel since you had 36k subs! Loving to see you slowly growing into a giant!
There is a discrepancy in what you're saying, Siraj. On the one hand, you say that you read 10-20 papers each week, but the steps that you follow might take up to 1-2 days, if you really want to understand a complicated topic. As I also prefer to understand deeply the covered topics, reading papers has always been a very big time commitment for me. If I wanted to read 5-10 papers a week... well, it would be a full-time job.
Thanks Sergey for pointing that out. One thing i think i should've mentioned is that for the papers that im mostly familiar with, i'll read them in a single pass. (like tweaking one or a few hyperparameters) whereas papers that are radically different require a 3-pass approach.
Also the goal isn't just to read a lot of papers. The goal is to achieve whatever you want to achieve. So for me, right now, I have a specific startup idea I want to implement; I'm not going to need to look at 20 papers a week. That's more Siraj's thing because it's his job to educate people on UA-cam and (I'm just spitballing) he just likes doing it
Wow, you have become the change what you are envisioning in the world to get transforma Siraj . Thanks a lot . It really means for fighting and creating hope for a family suffering from a dangerous decease ...so never give up dude ..you are our hope
whenever i just get more frustrated r struck in my research ..u r my best go to option to get inspired from... "#degree not needed" really i recall about u whenever im struggling for my research... such a light u r really ... keep going ....lots of love from INDIA...WATCHED this same so many times ....still it says something new to me...
Bro you said a very deep thought which very few people understand studies should be goal orientated like what you want to develop then study about that but in our education system it is opposite first we complete our studies and then try to develop or invent something which reduces the interest of students because they don't know why they are studying.
You make a good point but its difficult to formulate a education system thats "goal oriented" and works for millions of students. For example first year engineering or CSC students will learn calculus for a year (limits, differentiation, integration). Students will only use a part of what they learn depending on their specialization, but every topic has some application. How could we make this more goal oriented while still not losing the amount of content taught?
they could introduce the applications of calculus in their majors for example. In every engineering calculus is needed as derivations and understanding the formulas to be used. i doubt you could graduate engineering without understanding calculus
I knew there was a reason for it being difficult to keep up with you. You do that thing that I haven't gotten around to doing yet. Reading research papers. Bravo.
Thank you siraj... Your video is right on the spot..was looking for something like this..Can you also make one video a month to tell us what papers you found interesting and worth reading.Keep doing the good work
Siraj bruh, you're the bomb😂; smart, serious about life without taking it seriously. Been surrounded by way too many serious people in our science faculty, and I was basically one of the only totally unnecessary people😂😂😂. Love your videos too👍.
The most important part is to ask yourself whether you truely understand what you are reading. Especially when it comes to names and jargon. For example when it comes to complex Hilbert spaces, you should google what that is.
tl;dr Have a goal (something to build). This will guide your reading towards what is actually practical and relevant. 1. Skim. Title, Abstract, (Why should I care? What problems does this technique solve? Is it worth my time to keep analyzing this paper?) 2. Take notes on all the English, plots and graphs, look for stuff he already understands so he can put it in context. Read code and understand it through comments. Teaching it helps me (Siraj) understand it. a. Explain it on UA-cam to truly understand all the details. 3. Math: Write out derivations if you do this! Don't just read it. He'll recode it if it's really important to do so; this gives him a thorough understanding of what the tech does. Then he'll discuss with others to make sure he actually understands what it does If necessary, study the math on Khan Academy Ask for help! (stackoverflow.com, machinelearningmastery.com, etc.)
Hey Siraj! I can't thank you enough brother for those resources links to the awesome ML/Data papers. I wish if the majority of Professors in the Universities were at least half as interesting/motivating as you. Keep up the great work!
I think its in reference to Schmidhuber complaining that he does not get enough credit. See his post (and response from Yann) here: plus.google.com/100849856540000067209/posts/9BDtGwCDL7D
siraj, what was your math background when you first dive into machine learning and what advance math are you learning right now? abstract algebra, topology,...?
you mentioned Arxiv Sanity Preserver ... great concept. Here is one more I use that I find helpful if I want to dig deeper into the paper. Bring it up in Google Scholar and look at who cited it. There is a direct correlation between influential/importance/seminal and number of folks who cited it. Keep on truckin'
how do you do all those things you mentioned , 3 pass reading, github code parts, replicating the code, delving into the maths, creating jupyter notebooks, browsing the web about the article etc. if you read 20 papers a week!!!
Siraj is undoubtedly smart and competent. Even if he is able to do the depth x quantity that he specifies, broadly, i think others are not going to be able to replicate that process, or at least not for some time. I can imagine it taking a week to go through that Goodfellow paper to develop any real competence and understanding in its contents -- unless you have enough development in associated papers that overlap strong aspects of its contents. The "20 papers aspect" reflects speed and speed comes with more experience, heightened associations within that experience, culminating in the development of efficient processes; hence, a practicing researcher has the toolkit to facilitate speed. I think the goals stated in the video were wrong ... perhaps it is better to (a) identify some key papers representative of leading areas that are interesting, (b) with each paper, attempt to develop understanding, (c) and instead of 3 passes, make it 5, 8, 10, 15 passes. The goal should be to develop "columns of strength" by utilizing core papers to develop competency upon and then over time, seek to maximize speed, accelerating the number of papers viewed. As with everything a strong base needs development upon which to build breadth of understanding. However, there were many interesting ideas in the video, such as finding where others are "discussing" the paper so that gaps can be identified and answered. etc etc
I study a different subject, so some of these points do not apply to me, but I still think you brought up a few things which might be invaluable. I've never thought of actually looking for what other people are saying about it or that I might find a breakdown: this might save me some time. Also, definitely don't be afraid to reach out to researchers/people who might know more about the field you are studying!
thought the `have a goal' tip was a great tip - i write a paper every summer & my deadline is end of july. find this really useful for my discipline & motivation
thanks a lot for making this. I feel completely lost when reading papers and I never understand how people I know can create programs out of them. Always felt like I'm behind everyone else for not being able to do that.
In my studies I was in a course where we had a paper nearly every week. We were motivated to read it by answering a key question about an approach/algorithm/Problem occuring in the paper. Just by doing that, it feels like a minor success and then i already got a grasp of whats happening in the paper and want to read certain aspects more. Like Siraj said, having a goal in mind makes the difference
This is super cool. More videos walking through top research papers in-depth would be super beneficial. Walking through the GAN paper or the original Lecunn NN paper in one of your hour long videos would be a fantastic resource.
This is literally the question I had after coming across your channel. "How does he learn all this stuff???" Thanks for breaking down the process for us!
hahah :P I just discovered Reddit again, I never used it until you recommended it for research. didn't even know there was anything reliable on that website. thanks!
That he replaced complex hamilton space with complicated hamilton space, and logic gates with logic doors in his plagiarized paper, means he has never read a research paper
nice video Siraj! sometimes I feel like people make things overly complex in papers in order to flex their mental skills. I much prefer watching a tutorial talk like your videos or an animated video like 3blue1brown, but maybe that's asking too much. Anyways, keep making videos!
Mark Jay I've a little bit different opinion that yours. I think that kind of autors don't write in a easy way 'cause him haven't didactic, or him was stuck in a academic way of write from some academias. This make sense to you?
You have to understand that papers which are first of their kinds need to illustrate the proof(s) for their results. And sometimes these proofs are hard to express succinctly. You have to appreciate the mathematical notations of these proofs because they are the only source of reasoning that we can use to convince other people that we're not insane and why they should trust what we say to be true and trustworthy. And Siraj does us the favour to go through the papers and translate those equations into easy and approachable concepts so that we don't have to exert effort into understanding them ourselves.
That's usually known as the abstract. I agree with Hugo, many authors heavily inflate the importance and complexity of what they are actually doing. You get used to dividing by 1000 or whatever the number is after you've read enough in the field. Also, they aren't written for the audience of learning unfortunately. They are written for people who are already experts in the field to review. You can often get started just by collecting the top references at the end and reading the abstracts of the collection you come up with. You can often find issues or things they didn't cover or something you would like to explore further. This is your opportunity to develop the field, if it hasn't been done already. Great video Siraj!
Schmiduber has an immense contribution in the field of AI. Who cares that he doesn’t behave at events!? You, calling him out like that is not fair. He gave us LSTM, CTC and yes Prediction Minimization.
Thanks for the video man. i always find research papers kinda boring and kinda intimidating. But i really wanna read them on a regular basis. Will try to apply these techniques :)
thanks for the insights ... I use a similar method, except I work in two modes : goal directed and canvass mode (quasi-random). Which mode I use depends on what part of my brain has decided to work well. What the latter mode allows me to hopefully do is "run" into that strange, quirky paper that generates "out-of-the-box" ideas. I usually will find those papers in the application of AI, Machine Learning or Computer Vision. E.g. Biomedical fields, or Plant Science. Currently I am working on a concept : images as a language of textures (great approach for dealing with occlusion). Which led me to Oxford Robotics (textures) and CMU Graph Learning. Best in everything.
nothing more important than learning how to read the papers. many thanks
For a 2nd year bachelor student like me, who is worried whether he will be able to read these paper with high lvl maths into them, you gave a seriously outstanding breakdown on how to not get scared by papers and tackle them.
Thank you for that.
Loving this channel since you had 36k subs! Loving to see you slowly growing into a giant!
This is one of the best channels for AI, ML and BC. Thanks Siraj. Keep the great work!
@Siraj When we will have the video on how to plagarise papers?
There is a discrepancy in what you're saying, Siraj. On the one hand, you say that you read 10-20 papers each week, but the steps that you follow might take up to 1-2 days, if you really want to understand a complicated topic. As I also prefer to understand deeply the covered topics, reading papers has always been a very big time commitment for me. If I wanted to read 5-10 papers a week... well, it would be a full-time job.
Thanks Sergey for pointing that out. One thing i think i should've mentioned is that for the papers that im mostly familiar with, i'll read them in a single pass. (like tweaking one or a few hyperparameters) whereas papers that are radically different require a 3-pass approach.
that is the whole point
Correct
Sergey Antopolskiy exactly, “knowing physics” isn’t just understanding the concepts it’s being able to apply them
Also the goal isn't just to read a lot of papers. The goal is to achieve whatever you want to achieve. So for me, right now, I have a specific startup idea I want to implement; I'm not going to need to look at 20 papers a week. That's more Siraj's thing because it's his job to educate people on UA-cam and (I'm just spitballing) he just likes doing it
well, this didn't age well did it...
Why? (I don't watch his channel normally, so I don't know if anything happened in the last years)
Michi L. He plagiarized a research paper
Wow, you have become the change what you are envisioning in the world to get transforma Siraj . Thanks a lot . It really means for fighting and creating hope for a family suffering from a dangerous decease ...so never give up dude ..you are our hope
whenever i just get more frustrated r struck in my research ..u r my best go to option to get inspired from... "#degree not needed" really i recall about u whenever im struggling for my research... such a light u r really ... keep going ....lots of love from INDIA...WATCHED this same so many times ....still it says something new to me...
Bro you said a very deep thought which very few people understand studies should be goal orientated like what you want to develop then study about that but in our education system it is opposite first we complete our studies and then try to develop or invent something which reduces the interest of students because they don't know why they are studying.
You make a good point but its difficult to formulate a education system thats "goal oriented" and works for millions of students. For example first year engineering or CSC students will learn calculus for a year (limits, differentiation, integration). Students will only use a part of what they learn depending on their specialization, but every topic has some application. How could we make this more goal oriented while still not losing the amount of content taught?
they could introduce the applications of calculus in their majors for example. In every engineering calculus is needed as derivations and understanding the formulas to be used. i doubt you could graduate engineering without understanding calculus
@@devvv4616 Trust me, in India, you can. :P
you're right
This might be the one of the most important skill to have hence this might be the best video on the internet (at least for me). Good job!
You don't look like one but after watching your videos I can happily conclude that you're an extremely talented person. Appreciated.
I knew there was a reason for it being difficult to keep up with you. You do that thing that I haven't gotten around to doing yet. Reading research papers. Bravo.
Huh. Kinda awkward watching this video again in 2019.
I'm doind in 2020
He plagiarized a research paper and claimed it was his own work...
You are doing us a favour by making amazing videos on such topics which we need but none are available.
you are so humble.. thanks a lot
Thank you siraj... Your video is right on the spot..was looking for something like this..Can you also make one video a month to tell us what papers you found interesting and worth reading.Keep doing the good work
Amazing! I like the way you explain such hard concepts so easily in a fun way maintaining the humor at the same time.
I like humor also)
Explaining your approach for reading science paper : brillant idea and great video. Cheers, man!
Siraj bruh, you're the bomb😂; smart, serious about life without taking it seriously. Been surrounded by way too many serious people in our science faculty, and I was basically one of the only totally unnecessary people😂😂😂.
Love your videos too👍.
Siraj: No need to read papers, you can write papers with just two magic spells! and they are - ctrl+C and ctrl+V -----2019
For a degree?
Yeah, that's what he does.
I knew I would see this ctrl C ctrl V comment haha
maybe, maybe not
The most important part is to ask yourself whether you truely understand what you are reading. Especially when it comes to names and jargon. For example when it comes to complex Hilbert spaces, you should google what that is.
Siraj: I read papers with a GOAL.
UA-cam: Yes, Siraj, we get it.
Its a very helpful video. I have been facing a lot of trouble while reading research papers. I feel confident now. Thanks Siraj!!
Amazing. Thanks, really needed this.
It will be great if you also make a video on how to write a research paper.
tl;dr
Have a goal (something to build). This will guide your reading towards what is actually practical and relevant.
1. Skim. Title, Abstract, (Why should I care? What problems does this technique solve? Is it worth my time to keep analyzing this paper?)
2. Take notes on all the English, plots and graphs, look for stuff he already understands so he can put it in context. Read code and understand it through comments. Teaching it helps me (Siraj) understand it.
a. Explain it on UA-cam to truly understand all the details.
3. Math: Write out derivations if you do this! Don't just read it. He'll recode it if it's really important to do so; this gives him a thorough understanding of what the tech does. Then he'll discuss with others to make sure he actually understands what it does
If necessary, study the math on Khan Academy
Ask for help! (stackoverflow.com, machinelearningmastery.com, etc.)
Hi Siraj, can you please make a video on how to overcome meth addiction?
Start cocaine xD
Do some math you'll like it they're pretty close.
@@canesvenatici9588 you mean a and e? xD
Only users lose drugs
@@vishavjeetsingh7862 well of course, because if you are not a user you don't have any drugs to lose lol
i like this channel, havent seen anything like it. Very smart guy.
hey siraj.. please do make a video on how to overcome distractions and also how to gain interest in reading boring and tough but important notes
Thank you for the video. This video will go a long way to help me in my research work.
I like the way you make videos on the important things. Awesome. You've got something great here.
Hey Siraj! I can't thank you enough brother for those resources links to the awesome ML/Data papers. I wish if the majority of Professors in the Universities were at least half as interesting/motivating as you. Keep up the great work!
Why not Schmidhuber? He's one of the inventors of LSTMs and certainly a talented man. I don't understand why everybody suddenly started hating on him
I think its in reference to Schmidhuber complaining that he does not get enough credit. See his post (and response from Yann) here: plus.google.com/100849856540000067209/posts/9BDtGwCDL7D
Needed this video for a long time.Thanks Siraj..U are an inspiration..Keep it going..
You are my Hero Siraj 🤘. keep inspiring us.🙌🙌🙌🙌
siraj, what was your math background when you first dive into machine learning and what advance math are you learning right now? abstract algebra, topology,...?
You are the best! Awesome job
Huh.. watch this ua-cam.com/video/SHTOI0KtZnU/v-deo.html
I m really inspired. I will start reading following your tips.
Thank you so much for this beautiful video, reading a research paper in 3 steps really make sense !
Great video Siraj! Didn't know about Arxiv Sanity, but I love finding new resources to accelerate my learning. Thanks!
happy to help michael
Very helpful video for finding things to learn in Machine Learning, Thanks
i am sooo glad i found your page, thank you for making learning seem more interesting and the knowledge more attainable!
He's so good at reading research papers.
you mentioned Arxiv Sanity Preserver ... great concept. Here is one more I use that I find helpful if I want to dig deeper into the paper. Bring it up in Google Scholar and look at who cited it. There is a direct correlation between influential/importance/seminal and number of folks who cited it. Keep on truckin'
This is first vedio of your's and it is very good ..
how do you do all those things you mentioned , 3 pass reading, github code parts, replicating the code, delving into the maths, creating jupyter notebooks, browsing the web about the article etc. if you read 20 papers a week!!!
Siraj is undoubtedly smart and competent. Even if he is able to do the depth x quantity that he specifies, broadly, i think others are not going to be able to replicate that process, or at least not for some time. I can imagine it taking a week to go through that Goodfellow paper to develop any real competence and understanding in its contents -- unless you have enough development in associated papers that overlap strong aspects of its contents.
The "20 papers aspect" reflects speed and speed comes with more experience, heightened associations within that experience, culminating in the development of efficient processes; hence, a practicing researcher has the toolkit to facilitate speed.
I think the goals stated in the video were wrong ... perhaps it is better to (a) identify some key papers representative of leading areas that are interesting, (b) with each paper, attempt to develop understanding, (c) and instead of 3 passes, make it 5, 8, 10, 15 passes.
The goal should be to develop "columns of strength" by utilizing core papers to develop competency upon and then over time, seek to maximize speed, accelerating the number of papers viewed. As with everything a strong base needs development upon which to build breadth of understanding.
However, there were many interesting ideas in the video, such as finding where others are "discussing" the paper so that gaps can be identified and answered. etc etc
I study a different subject, so some of these points do not apply to me, but I still think you brought up a few things which might be invaluable. I've never thought of actually looking for what other people are saying about it or that I might find a breakdown: this might save me some time. Also, definitely don't be afraid to reach out to researchers/people who might know more about the field you are studying!
Thanks man, hello from Romania, have a nice day !!!
Thanks for posting this and sharing the resources, your energy and enthusiasm is great.
thought the `have a goal' tip was a great tip - i write a paper every summer & my deadline is end of july. find this really useful for my discipline & motivation
You are a genius is the motivation that counts!!!
“That resembles a game!? Alright, this is lit.” -Siraj Raval, 2018
He really helps starters
thanks a lot for making this. I feel completely lost when reading papers and I never understand how people I know can create programs out of them. Always felt like I'm behind everyone else for not being able to do that.
In my studies I was in a course where we had a paper nearly every week. We were motivated to read it by answering a key question about an approach/algorithm/Problem occuring in the paper. Just by doing that, it feels like a minor success and then i already got a grasp of whats happening in the paper and want to read certain aspects more. Like Siraj said, having a goal in mind makes the difference
I think the key question is the most important thing when you read the paper. And how to propose a meaningful question is difficult!
Simply Awesome!!! Simply Great!! Great work Maestro!
I thought you would never add this video. U r the best!
This is super cool. More videos walking through top research papers in-depth would be super beneficial. Walking through the GAN paper or the original Lecunn NN paper in one of your hour long videos would be a fantastic resource.
This is literally the question I had after coming across your channel. "How does he learn all this stuff???" Thanks for breaking down the process for us!
through plagiarism ✨
hahah :P I just discovered Reddit again, I never used it until you recommended it for research. didn't even know there was anything reliable on that website. thanks!
dude you r the real MVP
Great video, many thanks for your advice! Very useful for PhD-students :)
Plz make video on the how to start machine learning from the basic
This is an excellent video Siraj, thanks for your time as always.
happy to help
Well done! Godlike level.
Good job Siraj.
Great video! The 3 steps approach is good one! Thanks
loved it, I would definitely try this way because my earlier ways were pretty boring.
nice talk on machine learning
Thank you Siraj!
thanks for the consistently great vids siraj!
Great work Siraj. Something I have been looking forward to.
good to end the week with such a good message and insights! Keep doing good bro!!
thanks, professor Siraj
Thank you so much for this informative video
That he replaced complex hamilton space with complicated hamilton space, and logic gates with logic doors in his plagiarized paper, means he has never read a research paper
WOW. Great video man, and thanks for sharing your strategy as reading these papers can be so difficult I bought a book on how to do research.
nice video! it motivates me to finish my thesis right now
Step 4: I will make a UA-cam video based on the research paper.
Nice video Man!
awesome explanation
Thanks Siraj... This was really required.
Awesome! You should make a video or a list of your favorite papers. Maybe your top 10, 20?
nice video Siraj! sometimes I feel like people make things overly complex in papers in order to flex their mental skills. I much prefer watching a tutorial talk like your videos or an animated video like 3blue1brown, but maybe that's asking too much. Anyways, keep making videos!
Mark Jay I've a little bit different opinion that yours. I think that kind of autors don't write in a easy way 'cause him haven't didactic, or him was stuck in a academic way of write from some academias. This make sense to you?
You have to understand that papers which are first of their kinds need to illustrate the proof(s) for their results. And sometimes these proofs are hard to express succinctly. You have to appreciate the mathematical notations of these proofs because they are the only source of reasoning that we can use to convince other people that we're not insane and why they should trust what we say to be true and trustworthy. And Siraj does us the favour to go through the papers and translate those equations into easy and approachable concepts so that we don't have to exert effort into understanding them ourselves.
That's usually known as the abstract. I agree with Hugo, many authors heavily inflate the importance and complexity of what they are actually doing. You get used to dividing by 1000 or whatever the number is after you've read enough in the field. Also, they aren't written for the audience of learning unfortunately. They are written for people who are already experts in the field to review. You can often get started just by collecting the top references at the end and reading the abstracts of the collection you come up with. You can often find issues or things they didn't cover or something you would like to explore further. This is your opportunity to develop the field, if it hasn't been done already. Great video Siraj!
really cool video, how do you fight with math ?
Great video. Im trying to make my own user end software with machine learning algos
Schmiduber has an immense contribution in the field of AI. Who cares that he doesn’t behave at events!? You, calling him out like that is not fair. He gave us LSTM, CTC and yes Prediction Minimization.
One of your best.
That was awesome, I totally needed this!
Thanks for the video man. i always find research papers kinda boring and kinda intimidating. But i really wanna read them on a regular basis. Will try to apply these techniques :)
this is really helpful, thank you for uploading!
it also works for non-math/technical articles, thats awesome! :)
Super useful video Siraj, Thanks!
I love your attitude!!! thankssssss
Amazing! I felt the need for this while attempting to read white paper by Satoshi Nakamoto!
A UA-camr whom I need but don't deserve. 🙌
Super helpful man, thanks!
AMAZING Great Work!
Hi Siraj! Do you have any advice for newbies, for people that have never in their lives read a research paper and just getting started?
Thank you fellow ... really well done ...
i'm a simple man. i click on Siraj, my mind is blown
SCHRODINGER’S CAT I am even simpler man. I see Schroedinger's cat I get cancer
thanks for the insights ... I use a similar method, except I work in two modes : goal directed and canvass mode (quasi-random). Which mode I use depends on what part of my brain has decided to work well. What the latter mode allows me to hopefully do is "run" into that strange, quirky paper that generates "out-of-the-box" ideas. I usually will find those papers in the application of AI, Machine Learning or Computer Vision. E.g. Biomedical fields, or Plant Science. Currently I am working on a concept : images as a language of textures (great approach for dealing with occlusion). Which led me to Oxford Robotics (textures) and CMU Graph Learning. Best in everything.
yo man your video helps me a lot keep going. by the way i'm from Mongol