Machine Learning PhD Survival Guide 2021 | Advice on Topic Selection, Papers, Conferences & more!
Вставка
- Опубліковано 7 чер 2024
- #machinelearning #phd #howto
This video is advice for new PhD students in the field of Machine Learning in 2021 and after. The field has shifted dramatically in the last few years and navigating grad school can be very hard, especially when you're as clueless as I was when I started. The video is a personal recount of my mistakes and what I've learned from them. If you already have several published papers and know what to do, this video is not for you. However, if you are not even sure where to start, how to select a topic, or what goes in a paper, you might benefit from this video, because that's exactly how I felt.
Main Takeaways:
- Select niche topics rather than hype topics
- Write papers that can't be rejected
- Don't be discouraged by bad reviews
- Take reviewing & teaching seriously
- Keep up your focus
- Conferences are for networking
- Internships are great opportunities
- Team up with complementary skills
- Don't work too hard
OUTLINE:
0:00 - Intro & Overview
1:25 - Thesis Topic Selection
4:25 - How To Publish Papers
5:35 - Dealing With Reviewers
6:30 - How To Be A Reviewer
7:40 - Take Teaching Seriously
8:30 - Maintain Focus
10:20 - Navigating Conferences
12:40 - Internships
13:40 - Collaborations
14:55 - Don't Forget To Enjoy
Transcript: www.notion.so/Yannic-Kilcher-...
Credits to Lanz for editing
Links:
TabNine Code Completion (Referral): bit.ly/tabnine-yannick
UA-cam: / yannickilcher
Twitter: / ykilcher
Discord: / discord
BitChute: www.bitchute.com/channel/yann...
Minds: www.minds.com/ykilcher
Parler: parler.com/profile/YannicKilcher
LinkedIn: / yannic-kilcher-488534136
BiliBili: space.bilibili.com/1824646584
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: www.subscribestar.com/yannick...
Patreon: / yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n - Наука та технологія
OUTLINE:
0:00 - Intro & Overview
1:25 - Thesis Topic Selection
4:25 - How To Publish Papers
5:35 - Dealing With Reviewers
6:30 - How To Be A Reviewer
7:40 - Take Teaching Seriously
8:30 - Maintain Focus
10:20 - Navigating Conferences
12:40 - Internships
13:40 - Collaborations
14:55 - Don't Forget To Enjoy
wtf happened to the minecraft videos my dude????
Thank you for the video. I wanted to ask you about the post-doc vs corporate research opportunities.
The best advice I ever received in my PhD was to not treat my dissertation as a monolith. After a year of spinning my wheels, I broke it down into daily goals (e.g., write 2 pages on Chapter X each day this week). If I reached my goal at 2pm, I'd give myself the freedom to go to the movie or go hiking. No guilt. When I broke things down, I was much happier and more productive.
The second thing that I'd recommend is write to be understood. Far too many academics write in a style that is intentionally obfuscating - they literally don't want to be understood because most papers are not that exciting when you truly understand what they've done or they are just parroting others. If you aren't proud of what your doing, don't submit. At one conference, the lead reviewer told me that they enjoyed my "folksy' style. I was kind of insulted, but then they put it in for best paper. So, that was kind of cool.
Agreed. I've read some badly written papers where you cannot figure out what supposedly clever people are trying to convey. It's because they're using inappropriate English. or deliberately choosing to obscure the mediocrity of their work.
Top tip. If it's difficult to understand, it's likely the authors don't understand it either, or are covering up. Best to avoid ANY paper written that way. The best and the brightest are those who know how to communicate their ideas in a clear manner.
I see random papers by random niche professors not making hardly any sense. Then I see DeepMind researchers explain their research and I realize when you're happy (and make good money) you'll write cohesive papers. Maybe to be expected in academia with subpar salaries
I want to make a paper titled: "Reviewers are all you need".
Thanks u for speaking my mind 😊
Attention is all you get(ting)
Interestingly one ICLR-21 paper did this, and they reviewed the openreview. XD
😂
so funny but true lool
There’s actually almost no information online about this world of PhD. Somebody needs to investigate it like we investigate other anthropological phenomenon. Scientifically. That would be fascinating.
Very well put! I'm still an undergrad and I already wished there was a more complete review on graduate school life. I wonder what could push the right academics to study this.
Society is just PhDs all the way down
I want to add to "taking teaching seriously" part. I agree with all the things you said. To add to that, I would say teaching has one more benefit I see: you learn what you are teaching much better. This benefit should not be disregarded. Your comprehension of the topic increases as you teach.
Takeaways: Niche in and stay focused!
Holy production level increase haha I love the style. Keep it up!
Holy jeepers Yannic, those mid-sentence transitions are blowing my mind!
Top it off with the 80s drums and bonus BOOM at the start, well I'll be Yannic, you just put the cherry back in my pie.
thanks, Yannic I almost watched your every video on the transformer and I must tell I see you as a role model. I recently started my PhD at Ecole Normale Superieure, Paris and it is extremely helpful to gain better insights into how things work. If I were to pick my favourite video that would be your transformers video on Xl Net. Once again many thanks Yannic 👍
Nice nuggets of advice, Yannic!
However, you should have also covered the sinister side of PhD and academia in ML today.
The whole field has turned into an absurd fashion show and conferences are just vanity fairs where everyone is showing off their yet another fancy model which is not guaranteed to work in real-life scenarios.
How many papers have just selected a lucky random seed to get this ‘fantastic’ metric improvement?
How many actually report statistical significance for their results?
How many can actually explain what they’re doing to a high school graduate (hello Mr. Feynman!)?
I’ve personally reimplemented a few NeurIPS papers where models actually produce rubbish instead of the result they claim.
Don’t forget the toxic advisors who have no idea what they’re doing, yet still push you through the grinder to get a new paper every month.
Let’s be honest, academia nowadays is just a giant Petri dish for manipulative, threatening, deceitful, sociopathic, political sadists who abuse PhD students as their slaves.
To those applying for PhDs: go ahead if you want to find out how it feels to be in a Stanford prison experiment. Good luck!
Real and true. I'm facing all these problems and still don't know how to deal with them properly.
@@airazure2050 I think we should not focus on the problem, but the solution. Be different. Believe in yourself, not in the toxic comments of other people around you. Ignore your toxic advisor as much as possible, it is your PhD. The only reason they are doing this is because they are trying to protect themselves. They are successful in this flawed, f'ed up system and they want it to stay that way.
There's a reason why many breakthroughs in science came from people outside the academic world... So many papers published, yet so little knowledge gained
@@airazure2050 You are not wrong, especially the fasion thing and the funny part is that ML research to a large degree assumes that good accuracy on MNIST/CIFAR/IMNET translates to any kind of real world impact. It sometimes does but more often than not it probably does the opposite and discredits research that actually would work in the real world but may not reach SOTA on those toy datasets.
However, one big point I absolutely don't agree is with the advisors and supervisors. I've seen both and I've seen the same advisers being perceived as the complete opposite by different people. I think what's crucial is that you personally get along with your advisor, then everything is a thousand times better and it's probably worth to quit early on if you do not.
Great video Yannic. Thank you for sharing the insights. Would love a video suggesting those who aren't into PhD program yet. To tackle questions like doing PhD vs not doing, the importance of doing a PhD, examples of successful ML practitioners/individuals without PhD.
This was a fantastic video. I am a year into my PhD and have made almost all these mistakes myself already. So relatable. Really, really well done!
can we talk i have some questions and i want advance people opinion ?
the right video at the right time. I'm starting an end-study internship about forecasting and data streaming and it will be surely followed up by a thesis. thanks for the tips.
This video is clearly a great inspiration. Loved every aspect of it.
This is such a good video, I made similar mistakes myself and wish I had done it a little bit differently. But I hope all junior PhD see this before continuing their journey!
Thank you so much! As I failed/missed so many things during my masters years, I seriously dont want to fail again during my Ph.D years. Niche topics...Ill keep that in mind
As a Google Researcher, I agree with the conference assessment that your precious time at a conference is best spent at the poster sessions, meet and talk with other researchers, going to workshops, and attending events over watching the main event talks in person. Especially how tough it is to get from one talk to another at big conferences like NeurIPS, ICML etc.
You are so humble, i respect you a lot because i learn so much paper from you
Dude, beautiful video, I can relate to all points!
Love the mad editing skillz
Great to niche in and nice editing video, Yannic
Thx
I started a PhD in machine learning and metagenomics literally yesterday, the timing could not have been better!
can we talk i have some questions and i want advance people opinion ?
What did you study before to get into that combination of fields?
@@Sam-py9qq My undergrad degree was in bioinformatics, which is a mix of computer science and biology. That is also the field I worked in for 4 years before coming back to school. That said, most people in my lab have an electrical engineering background and pick up the biology when needed. I taught myself all of the machine learning that I know so far (but will be taking classes when available).
@@navinbondade5365 I am not advanced in the field of machine learning (yet) and therefore cannot really give any good advice for that field :(
@@Welank My past is quite similar to you, I had done a diploma in Civil Engineering after secondary high school and then change my career path doing Computer Science in Undergrad instead of Civil, I also teach myself most of ML, DL, NLP, and Computer Vision I will finish my undergrad this year and aiming for MS in ML. Can you share your LinkedIn profile link?
Thanks for sharing. It's truly the fact that I have to always remind myself: 'don't play with deeper models! '
I was a phd student in 2011 to 2017 it is soooo hard . If only I could see your video at this time it would help a lot !
I watched this video one tear ago. Tomorrow I start my PhD and needed to rewatch it again❤
Thank you for sharing! I just accept my PhD offer in Computer Science last week! Again, thanks for your advice on topic selection and conference navigation.
how's it going?
I finished a PhD in machine learning in 2018 in São Paulo, Brazil. My experiences are very similar to those reported by Yannic Kilcher. Congratulations on the initiative to share them here for a larger audience, instead of just for a few students.
hi sir i am looking for phd in machine leanrning please give me your guidance i need your help
Masterpiece, thanks for the advice!
VIDEO EDITING SKILL LVL GODLIKE!!! Very informative and very big THANKS for this. new subscriber here
Great guide, thank you very much for making it!
Truly many thanks for the advice. :)
cool editing!
Completely agree with the teaching bit. It can result in pretty much a paper machine. I'm in an area where machine learning wasn't a thing in the curriculum at all. I created a lecture where students have to do projects at the end and some of them are incredibly creative, some of these can be published. Some of the students went on to become master students for me, some moved to other institutes but mentioned me to their supervisors and I'm now in contact with them, working together on various topics. Some students went on to do master theses in companies, which now also contact me for advice.
There's really zero reason to slack at teaching. Doing a great job is a huge time investment, but the results far outweigh that.
Such a great video
I was wondering if you could make a video on:
Why to do/ not to do a PhD in ML =))
Watching this is making me consider grad school. Awesome video
Wow, this is gold!
I had a student who was so proud that he really didn’t know that he didn’t know how to choose topics…
granted, probably no one knows for sure what would be a good topic. but being extremely confident of one’s choice despite whatever others say is… admirable…
First semester in, so far so good. Thanks for video, never though conferences like comiccon for reserchers, in a way that only interesting thing there is ppl.
Thanks for your sharing. This is my first PHD year.
Thank you for this video. Should have had that 3.5 years ago.
Coming from an "applied" field, we have our "niche" from the beginning. But woe betide you, as soon as the real machine learning people find your niche, too :D
as someone who is not a PhD, I found this very insightful into what skills a PhD might acquire during the course of the program.
Thanks for your valuable insights.
Very good advice for newcomers!
My 2 cents regarding "Reviewers will try to find all reasons to reject your paper" -> yes, but that depends on how interesting and creative the story you present in your paper really is. If it is just an incremental boring/trivial/expected contribution, reviewers will be more likely to attack it from various angles compared to the case where the reviewers get the overall perspective of a nice/interesting/intriguing/non-trivial story. However, what ultimately gets you to be more and more creative in your ideas are actually papers and talks (far) outside of your niche area.
Thanks for this video!
thanks a lot for the guide !
This video is awesome. Thumbs Up.
Oops, first point and I already failed - my supervisors have no idea what I am doing. Great advice though, thanks a lot!
Great advice! I wish I saw this video in back 2012. :)
Great advice, especially on distractions that masquerade as useful activities! Thank you.
I wish I can give multiple likes to this video!
I really, really like that you're trying out these new things with your channel. The editing in of visuals that fit with the script really help with getting your point across. The use of music was very good. And of course the message that you're trying to put across in this video has lots of value to many people.
If I could do some constructive criticism, though, , it would be the backgrounds that have nothing to with what's being said. I found myself looking around the minecraft scenes, trying to find some connection with them and what you're saying instead of focusing on you.
I also see you recorded your script in multiple different locations which is good as it add variety, but when the scene switched I was subconsciously expecting something to change. For example, the theme could change, and the different locations could be "outside me #1 is what NOT to do", "outside me #2 is what TO do", and "greenscreen me is tying the script together"
Fantastic work, I like it.
I love You for this Wonderful Video...😍
I'm in the 3rd year of my Ph.D. and still a lot to take from this. Also could you please make a video regarding careers after Ph.D. and do's and do not's? How to get into a research scientist position?
I agree that focusing on one topic is important, but only if you are 100% sure that you really love what you are doing. I see so many people (including myself) getting into a topic during their phd because someone told them to, or theres a project that they get money from etc. And in the end they are lucky if they could finish or at least not hate the whole academia. So if you feel that you wanna go and see that interesting lecture think first, is it because you cant focus or because you hate what you are doing (this excludes the final weeks of you writing a paper, cause during that time you gonna hate it anyways :) .
Very interesting, thank you!
Great and valuable ✌️🌿
Hello Yannic, Thank you for these great tips! I am doing my Data Science PhD at Capitol Technology University. My dissertation topic is on using NLP to classify disease from clinical notes. I have a questions about theoretical and conceptual framework. I understand the definitions for them, but I do not how to fit them into my research. Capitol requires a framework section in the chapter 1. I have looked at other dissertations from Capitol, but none were on NLP or on clinical notes. Other NLP dissertations I looked up don't have a framework section.
Do you know of any theoretical framework that I can include in my research? The only one I found for NLP is the NLPIR framework.
Man, the montage and effects in this video are some next level CGI AI shit
I am interested in knowing how the reviewing ecosystem works, particularly the conference system (NIPS,ICML,AISTATS,UAI, etc). It looks like everyone (student, or faculty) has been frustrated with the reviews, but I often saw some groups have 5 nips/icml a year. The double-blind was meant good in old time, but reviewers easily knew who you are. There are many unsaid things in the top of ecosystem.
I am a Ph.D. student from China. Ph.D. students in machine learning are under greater pressure because there are so many of them in China. I hope to find some foreign friends to study together, maybe we can generate more inspiration.
Cool, man ! I want to get into a PhD prgram, currently doing my bachelors in India. Wanna work, study together and gain some more insight into the field. Can we connect?
I am actually doing PhD in Ai in south Korea. But here do not expect much help from your labmates or advisor. Maybe it is only the cases I know.
Great video
It was good. Perhaps would make more sense when I am actually doing Phd
Thank you Yannic !
Yannic, can you make a full video on thesis topics?
Has the point regarding transformers not being the hype 2 years later aged well? I genuinely curious: I know that most LLMs are based on them, but do we still have active research on the architecture per se ?
Hi Yannic, your pieces of advice are very helpful. At 5min10, how many papers should I read ( a range?) to have a good senses of my topic before I start to write my paper ? Thank you.
I am not going for PhD but still end up watching your video... Because i learn something new whenever I watch your videos...
But yes, i am going for masters... And if you can give some advice on that... It will be really helpful... 😁
And thanks for such useful videos...
Same here. It would be really helpfull if you could make a video about your Master :)
My two cents:
0. University is actually a good time to diverge. If you're not so sure about what it is that you want to pursue after your M.Sc., then follow subjects that you think are interesting; don't be afraid to fail. It's generally also a good idea to just register for more than the mandatory amount of credits per semester. It's common to drop some subjects as they turn out to be different than what you expected.
1. Find a group of people that are like-minded to work with during projects. There's nothing worse than half of your project group not wanting to put any effort in.
2. Establish friendships/connections. In x years from now, some of your fellow classmates will have reputable careers (maybe it will be you). Adding these people on platforms such as LinkedIn will allow you to follow their career path and learn from them. On top of that, if you are ever in need of a job, they might be able to help you (and vice versa) =).
3. Actively partake in lectures and build relationships with the professors of subjects that you like. Read about their work and see if you have anything that you would like to work on. Once you do, you have basically already found your supervisor for your thesis and professors can generally also help you with the subject. Make sure to tell them what your goals are (e.g. do you want to publish your work? If so, tell them!). A thesis can generally also be a good start for your Ph.D. if that is what you feel like pursuing :)!
4. Never go for the easy subjects. This can be tempting, but in the end, you have to think about it this way: If I were to become a Machine Learning Engineer/Data Scientist/whatever, how can I make sure that I can provide value for the company that I work for? This 'company' can also be the university if you want to pursue a Ph.D.
@@M0481 thank you so much... This is really helpful... And will surely help me in my journey... :)
@@ProgrammingCradle You're very welcome. Good luck and most of all: have fun!!
good choice. i was thinking that exactly this is missing: i.e. don’t do it 😛
Even though I am doing master, but loved listening this topic 😍.
Hi,This video is very helpful. Can you make a video on research at Masters level ?
can you pls explain the universal approximation theorem and its proof for depth case in an somewhat visual and intuitive way?
7:40 that is a lot of wind :D
Great talk
Much helpful. I'll be back to give feedback after 2.5 years when I finish my ML PhD... 😀
How's it going now? Any feedback?
Thank you!
Thanks so much! What would be your take on a part-time PhD though?
Yannic, can you please let me know the soundtrack you used towards the end? Thanks.
Perfect!
Yes, all the things you say are relevant to any PhD (not just ML)
- Be dilligent with a review (unless the paper is crappy. Yes, there are not just crappy reviewers but also crappy papers). Apart from that reviews are great. You get to read new research results before anyone else in the world. How awesome is that?
- Be focussed
- Try to publish in high level conferences/journals...don't go for a shotgun approach and just publish anywhere. That might hurt you more than it will help.
- Go to conferences. Even if you only have a poster. Network. a PhD isn't the end of your career and these are the people you will keep meeting. The number of people in a specific field are smaller than you think at first.
- Learn to read papers. Specifically learn to read what is *not* in there. Anything that's not in there - even if it seems like an obvious task to you - wasn't done.
...oh, and start writing on your thesis early, or you'll be chugging coffee like there's no tomorrow towards the end.
How does Machine Learning research make systematic improvement to model architectures? Because sometimes I feel it's either: 1) blackbox hyperparameter optimization; or 2) biological analogy/inspiration, hey lets try that see if it works. It feels neither case accumulates knowledge that can be used to make systematic improvement to model architectures.
"Focus on your research rather than other distractions"
proceed to produce lengthly youtube videos :)
I have been reviewing a few papers indexed in SCIE/WOS and it is credited in our University if I did good reviews. Its also a way for getting promoted.
I've just realized that I've never saw you without that glass.
just noticed - currently 5238 views - 582 likes - 0 diskes
=> I'm not the only one to really enjoy the channel !!!
I am seriously going to consider making a lot of money as an engineer and then do autonomous research with my savings. I am lucky enough to have pragmatic research interests so I can't really care about huge models that could never be trained on a machine I can afford. Anyways, there is a lot of theoretical foundations to lay before we get to understand neural architecture search to the point where small neural networks have any use in the real world. The compressive problem solving aspect is so much entangled with the most fundamental computer science problems that I know it can keep me busy for life. If I can't find a revolutionary idea there will be no regrets because that's the most likely result to expect anyway.
Niche is a somewhat relative thing: human knowledge has grown enough that any topic you choose, there is a rabbit hole associated with it constructed by many contemporary researchers.
I work in a "niche" topic within optimization (Multi-Objective Optimization), and guess what, I don't think it's a niche topic. :D
How's work life at Meta? How much does it pay for RS?
What about finding niche topics? Any recommendations on going about this?
I regret my decision not to get a PhD in ML slightly less now :)
Thank you
Interesting that almost nobody ever talks about the "purpose of a PhD". What is it?
In ai ,things changes fast. What is the difference between a research in ai domain and any other research in other areas ( like engineering or management)?
Is distributional Reinforcement Learning a niche topic ....the citations are scaled by 1000 to normal Reinforcement learning
I am asking this for my friend he is very confused, he has a good IT company, good work and is making good money, should he do PHD?
Hearing these tips makes me think that the underlying system for getting a PHD is really really bad.
Anyone know the name of the background music?
hey need a bit of help i am a graduated chemical engineer and now thinking to study further in machine learning and data science but i almost know nothing of machine learning and i have to submit a research proposal in machine learning or data science can someone help me to chose a topic or how to find a topi thanks
16:06 That Univ is EPFL right?
What's the name of the song at the end?
Best Advice: Take it a bit easier.... maybe you will have half a year longer, who cares, you only get to do a Ph.D. once. !!
Thanks Yannic !!
(this would help a lot of fellow students)
How do you get into a good Phd program in Ai and ML?