Superb presentation you made. Simply beautiful. The transformation from a distributional bar graph to a DNA strand ... amazing. So many designs and thinkings imbued throughout the presentation. Also, the color choice is great. To be skilled at both the science and the presentation is quite rare IMHO.
Thank you! I really appreciate it. As for the design of it, i just wanted to keep it clean and simple. Also since i made everything in Keynote i took advantage of it's "magic move" effect (that's what's happening there in that slide) :)
This video topic on LDA modeling is the best layman's presentation thus far, but even more so applicable in the recent state of AI modeling. Thank you dearly.
Instantly can see, when someone is talking about a topic in which he was involved himself. Great job! You definitely know what you are talking about. “If you can't explain it to a six year old, you don't understand it yourself.” ― Albert Einstein.
Straightforward and elegant explanation. I'm recently venturing into topic modelling for my research work and I was a little bit confused with the differences between keyword extraction and topic modelling and I stacked up many articles to read to learn more about that. However, thanks to my time here, I saved a whole lot of effort and time just by watching this video. This seems to be a good path to follow for splitting documents into conceptual and meaningful segments. Thanks, once again.
Thank you Ufuoma! In terms of reading, those ones are my favourites: pdfs.semanticscholar.org/529d/7107b9a6c7862b0536236a210611fd04261a.pdf menome.com/wp/wp-content/uploads/2014/12/Blei2011.pdf legacydirs.umiacs.umd.edu/~jbg/docs/nips2009-rtl.pdf psiexp.ss.uci.edu/research/papers/Griffiths_Steyvers_Tenenbaum_2007.pdf www.jmlr.org/papers/volume3/blei03a/blei03a.pdf
It's not only the clear explanations but also useful preprocessing tips from the speaker's prior experience that makes this video so useful! Loved watching it :-)
Well articulated tutorial. The thing I really like about this is, it actually has an industry application and that changes everything. Loved it, thank you for taking the time out to do this.
Man, this video is more than amazing !!! Thank you very much for that simple delivery of content and the amazing preparation of the slides/video itself !
This was very useful. I noticed that you don't have any other videos. I hope you start posting again. Maybe make a series about building a recommendation model from scratch. 😊
sorry to be off topic but does someone know of a way to get back into an Instagram account?? I was dumb lost the account password. I would appreciate any tips you can give me
@Abram Travis thanks for your reply. I got to the site thru google and Im in the hacking process atm. I see it takes quite some time so I will get back to you later with my results.
I have never commented on youtube videos before, but I got to say - you are a genius explainer. Loved the presentation - so elegant, thank you so much!
Thanks @Andrius for this awesome article. The crisp and elegant presentation with a practical use case scenario is a treat to be relished. Shared it with my friends as well on Twitter.
This is one of the best talks i have heard on LDA. It's well explained and helps a lot of setting the ground right for someone to start exploring this space. Thank you so much :-)
Thank you so much for the kind words. That video was actually the last thing i did while being a datascientist. And it was about a topic i was working with for a few years. So i'm very glad it came out the way it did and that people find it interesting and/or useful. It makes everything worthwhile. :)
Hey Andrius, great job explaining the topic modeling concept and relating it with the use-case of magazines. I really enjoyed the presentation, the graphics, video, and the entire layout plan. Kudos. Once again, thank you for posting it here.
What an amazing presentation Andrius. Very well explained and nicely crafted. Clearly demonstrates your deep understanding of LDA and fantastic communication skills!
This was great - very well done, thank you. You probably already know that between 15:30 and 16:15 you are talking about a slide that isn't being shown, and you only get a glimpse of it as you transition to the next slide (around 16:18) to the "put them on a space/simplex" visual. But, overall, just fantastic! MAKE MORE VIDEOS, Andrius!
Thank you! Happy to hear you liked it :) You are so right, there's a glitch there :( I dont how it happened since it wasn't like this before. I'll see what i can do about this. Worst case scenario - i just have to re-upload the video again. Thank you for flagging this!
Phenomenal, sharing with some colleagues! This is absolutely fantastic, I really appreciate this. The audio alone would have helped me a ton, and this is visually beautiful as well!
Thanks for the comment! Glad to hear you found it useful. Gensim is great! Made my life so much easier when i was a Data Scientist working with Natural Language Processing. They also have some cool stuff on Deep Learning! :)
Do you have any idea about how to train domain specific model like finance or restaurant? I think we can use wikipedia data but I have no idea on how to parse domain specific data from wiki dump.
Excellent Job ! On my top 5 of UA-cam presentations so far ! So clear and detailed at the same time. I deeply understand LDA with your explanations and the process of using it for the purpose of document similarity computation task. And I agree, the animations and visuals are perfect :-) Great Job again (I know, I've already said it)
Thank you Amelie. Super happy to know that you liked it :) the whole thing was done in Keynote (since I'm working on a Mac) and if you have any detailed questions on how I did this or that, please let me know. will be happy to help
Thanks so much, Arujinho. If there's any part of the presentation that I can break into "how exactly I did this or that step by step" please let me know. Will more than happy to share :)
@@andriusknispelis8627 I consider your presentation near perfect and more in detail about the process of LDA is probably left to the auditor to inform himself. This leeds me to a question: Which source do you recommend do dive a little bit deeper into the mathematical background of the process - especially considering a researcher that is rather "mediocre" in probabilstics ;-)
When it comes to LDA I'd definitely recommend to watch/read anything from David M. Blei (the author himself) One of his video lectures (1 hour 20 min long) ua-cam.com/video/DDq3OVp9dNA/v-deo.html When I studied it I haven't watched the videos myself, I just read articles. Here's one I can recommend from him: www.cs.columbia.edu/~blei/papers/Blei2012.pdf I also like this one very much, also I think the title is spot on accurate :) Reading Tea Leaves: How Humans Interpret Topic Models pdfs.semanticscholar.org/3a99/da22b1658695d95a764169e030cc40e2fb95.pdf
Wow! Thank you, this was so immensely helpful, with a real-life example used to illustrate the whole way. Even the things that were highly technical were grounded in the ASDA magazine example and it really helped my comprehension, because everything I'd read before this was very abstract. Keep the videos coming!!
Thank you! I have to admit that reason I chose ASDA magazine was because I like their covers :). And the reason I went for that particular issue is because of the green color, since it went well with the rest of the slides :)
Awesome Job Andrius. Great explanation and visuals. Very professional as well as effective. I've been using an implementation of LDA in R for a project, but had problems understanding the theory - this really helped.
Thanks Wes! Glad to hear you liked it and even happier that you found it useful. Makes it all worthwhile :) It was definitely my most used machine learning tool while i was a data scientist at issuu.
Dear Andrius Knispelis I would be grateful if you can share with me the source code for this implementation. or atleast help me to access the source code in gensim. Regards, Omar
Compact, crisp and strong narrative video presentation...I watched it only 2 times and understand the process thoroughly....1 question, just to get your insight..Is LDA can be combined with systematic literature review protocol (SLR) and the produced model in LDA is similar with structural equation modeling (SEM) model?
THANK you man! :) Hehe, that 2 was NOT a coincidense. It was really counting on somebody noticing it. And it made my day when i read your comment :) Cheers!
Thanks Sam! Glad to know you enjoyed it! I'd argue that perhaps the creative part of it was not using LDA topics as some sort of final classification, but instead as a fingerprint and then clustering those fingerprints separately outside of LDA.
Thanks for sharing the video. very informative. Question? LDA returns me a score for each topic when I infer a new document? how do you convert that score into the fingerprint?
hi Saurabh, that probability distribution that LDA gives you IS a "fingerprint", or "DNA", or "whatever_other_term_you_can_come_up_with". Basically, it's just another way of saying that it gives us something unique for each document.
Hi Andrius, thank you so much. This is a really amazing video, it's very well explained and helped me with my current project. Please make more videos!! Thanks.
It was one of the best presentations I came across. A good video. Andrius - Can you guide, on how you have created the dashboard showing words left, not in lda, in lda, unique. Also, what are the graphs at the top right of the presentation?
THANKS! I really appreciate it :) Happy you liked it. OK, so in the top right there are three graphs there (yellow, green and blue). Yellow - the similarity (JSD from 0 to 1), so i can see both the size of the neighbourhood and how quickly it dissolves into the rest of the documents (the slope of the curve). If it's a newspaper with many themes, it dissolves slowly, if it's a very concrete niche magazine - that curve is much more sharp. Green and Blue are both showing the same documents as in yellow one in the same order, but Y axis is showing the number of words in there. Green is total words (same as in "words left" on a grey barchart a bit to the left), Blue is unique words. I wanted to see how the LDA similarity relates to the number of words in a document. All three graphs only show the top 300 neighbours. First barchart, the one with all the colors, is simply taking all words from a document and matching them with a buch of lists. I had lists with city names, country names, people names, and so on. I have removed all of those words, only left the ones marked in white color there (in the top) The second barchart starts where the first one ends - first number is how many words are still left in the document after removing ones that triggered the stoplists. Then i broke that first number in two parts: a) words that were not in LDA model, and words that were. Then final number is how many of those were unique. So i know it's not just a several words being repeated a lot. Oh, and all those graphs were created using Python with Matplotlib package, and producing an HTML file. I was running hundrends of these tests, each generating an HTML file with a bunch of magazines, so i can easily browse through and see how it looks :)
thanks Richard! Making it clear was indeed one of my main goals here. I made this video right when i was making a shift from Data Science into Product Management. And I used it as one of my "portfolio items" since I wanted to show that i can take something technical and explain it. That was one of the reasons how this video came to be at that particular time. Happy to hear you liked it :)
Hey great presentation Andrius! Just curious are you planning to make more videos of different NLP models or ML models in general? The ideas are well explained with sufficient details and the way you organized and presented is awesome; it would be great if you could make more of these and share your wisdom with the rest!
Hi Shum, happy to hear that you enjoyed it. When I was a Data Scientist back at issuu, LDA was the thing I worked on basically daily, so I got to know it pretty well. Which is why I made the video on it. Sadly, I can not say that I'm familiar with other NLP models in the same way... So if I make more videos it will probably be about something else :). I'm thinking doing a little series on how to build your CV and something more on visualisation and/or presentations.
Superb presentation you made. Simply beautiful.
The transformation from a distributional bar graph to a DNA strand ... amazing. So many designs and thinkings imbued throughout the presentation. Also, the color choice is great.
To be skilled at both the science and the presentation is quite rare IMHO.
Thank you! I really appreciate it.
As for the design of it, i just wanted to keep it clean and simple. Also since i made everything in Keynote i took advantage of it's "magic move" effect (that's what's happening there in that slide) :)
Among the best presentation I ever encountered. Simply amazing!
Really an amazing presentation! It helped me a lot to understand LDA topic models thouroughly, thank you!
The best presentation I’ve seen for LDA’s and most other themes. Outstanding work thank you for producing it!
This video topic on LDA modeling is the best layman's presentation thus far, but even more so applicable in the recent state of AI modeling.
Thank you dearly.
thank you so much, happy you liked it!
By far the best mid-to-high-level explanation of LDA models I have come across. Thank you!!
Thank you Andrius Knispelis. I spent a lot of time on understanding LDA but your presentation gives the complete picture of LDA.
Instantly can see, when someone is talking about a topic in which he was involved himself. Great job! You definitely know what you are talking about.
“If you can't explain it to a six year old, you don't understand it yourself.” ― Albert Einstein.
I have gone through a handful of videos on LDA but this one is by far the best. Thanks Andrius for taking the pain to prepare this one for us.
Thank you Ayan! Happy that you liked it and humbled by your comment. Makes it all worthwhile! :)
Wow, you are a genius. You were able to explain such a complex topic in 20 minutes with breathtaking graphics!
Thank you soooooooooooooo much. The best topic modelling presentation ever!!!!!! Many thanks from a linguistic student.
Straightforward and elegant explanation. I'm recently venturing into topic modelling for my research work and I was a little bit confused with the differences between keyword extraction and topic modelling and I stacked up many articles to read to learn more about that. However, thanks to my time here, I saved a whole lot of effort and time just by watching this video. This seems to be a good path to follow for splitting documents into conceptual and meaningful segments. Thanks, once again.
Thank you Ufuoma!
In terms of reading, those ones are my favourites:
pdfs.semanticscholar.org/529d/7107b9a6c7862b0536236a210611fd04261a.pdf
menome.com/wp/wp-content/uploads/2014/12/Blei2011.pdf
legacydirs.umiacs.umd.edu/~jbg/docs/nips2009-rtl.pdf
psiexp.ss.uci.edu/research/papers/Griffiths_Steyvers_Tenenbaum_2007.pdf
www.jmlr.org/papers/volume3/blei03a/blei03a.pdf
Thanks. I'll check them out.
This tutorial is very much helpful to me to learn the LDA model as a beginner. And the presentation is excellent, clean, precise.
thank you so much for making such a complex concept relatively easy to comprehend
This is the highest quality video on LDA Topic Models.
Thank you Leo! I really really appreciate it!
One of the best presentations I've ever seen ..
Thanks
It's not only the clear explanations but also useful preprocessing tips from the speaker's prior experience that makes this video so useful! Loved watching it :-)
This is great, clear, and practical. Excellent package in 21 minutes. Thank you!
Well articulated tutorial. The thing I really like about this is, it actually has an industry application and that changes everything. Loved it, thank you for taking the time out to do this.
Thanks you, Nischal! It seems my time was well spent then :) THANKS
Man, this video is more than amazing !!!
Thank you very much for that simple delivery of content and the amazing preparation of the slides/video itself !
Hey Abdelrahman! Happy you liked it :) I really enjoyed making it
Very well explained, the best LDA explanation I found so far.
Thanks Maurizio, glad you liked it :)
This was very useful. I noticed that you don't have any other videos. I hope you start posting again. Maybe make a series about building a recommendation model from scratch. 😊
My goodness! That's one heck of a presentation. Great one Andrius.
sorry to be off topic but does someone know of a way to get back into an Instagram account??
I was dumb lost the account password. I would appreciate any tips you can give me
@Terrance Daxton Instablaster :)
@Abram Travis thanks for your reply. I got to the site thru google and Im in the hacking process atm.
I see it takes quite some time so I will get back to you later with my results.
@Abram Travis it worked and I now got access to my account again. I'm so happy!
Thank you so much you really help me out :D
@Terrance Daxton You are welcome =)
Superb explaination. Thanks a lot Andrius.
Man I loved this video. It helped me so much!! Really apreciated it. Now my master degree is on the right track once again!!!
Very well explained. It's pretty useful in our research. Now I know what is LDA's all about. Thanks!
Glad you liked it, Amelito! :)
I have never commented on youtube videos before, but I got to say - you are a genius explainer. Loved the presentation - so elegant, thank you so much!
Thank you Akshay! I'm very very happy to hear that you liked it. Cheers!
Really clear and concise way to explain this complex topic. Thanks
This is really great! Love the excellent visualization and methodical explanation.
Thank you very much for explaining a complex topic like LDA in very simple and intutive way.
You're most welcome, Tanveer! Thanks for watching.
Thanks @Andrius for this awesome article. The crisp and elegant presentation with a practical use case scenario is a treat to be relished. Shared it with my friends as well on Twitter.
Thanks Abhishek. Glad to hear that you liked it and super thanks for the share.
I already retweeted your tweet :)
An excellent video. Filled-in some unknowns / uncertainty. Thank you for taking the time to post.
No worries Kent! my pleasure :)
This is one of the best talks i have heard on LDA. It's well explained and helps a lot of setting the ground right for someone to start exploring this space. Thank you so much :-)
Thank you so much for the kind words. That video was actually the last thing i did while being a datascientist. And it was about a topic i was working with for a few years. So i'm very glad it came out the way it did and that people find it interesting and/or useful. It makes everything worthwhile. :)
Hey Andrius, great job explaining the topic modeling concept and relating it with the use-case of magazines. I really enjoyed the presentation, the graphics, video, and the entire layout plan. Kudos. Once again, thank you for posting it here.
This presentation is so much informative. Thanks!
Very nice intuitive description with useful graphics and enough detail to be very useful.
Wow, this is an excellent video explaining topic modelling, LDA and the use case. Great work! Thank you!
You are most welcome, Yu Xuan Tay! Glad to hear you found it useful :)
What an amazing presentation Andrius. Very well explained and nicely crafted. Clearly demonstrates your deep understanding of LDA and fantastic communication skills!
Thank you Alok, glad to know you liked it!
Thank you.. This was a real clear high level overview. Will tell with the many hours of frustration that will coming my way with LDA
Hey! Thanks for watching and i'm glad you liked it! :)
Awesome video......and delivering the topic precisely....I have never seen as good as this presentation ......thank you
Thanks man! :)
Super happy to hear you liked it. Was definitely fun making it.
It has given good insight both to theory and application and the way you correlated it with DNA.... amazing
usually i dont comment on videos, but this is such a great explanation of LDA! intuitive, interesting and visually appealing! keep up the good work
Thanks a lot Steve! Glad to hear you liked it. :)
Wow! Blown over by the video. It was easy to follow and gained a lot of information for implementing my model. Thank you
This was great - very well done, thank you. You probably already know that between 15:30 and 16:15 you are talking about a slide that isn't being shown, and you only get a glimpse of it as you transition to the next slide (around 16:18) to the "put them on a space/simplex" visual. But, overall, just fantastic! MAKE MORE VIDEOS, Andrius!
Thank you! Happy to hear you liked it :)
You are so right, there's a glitch there :( I dont how it happened since it wasn't like this before. I'll see what i can do about this. Worst case scenario - i just have to re-upload the video again.
Thank you for flagging this!
a proof you can see here vimeo.com/140431085 that this glich should NOT be here :)
OMG literally every single word you said in this video is super helpful to me. Thanks a ton!!
Thanks Peeta! :) Happy you liked it!
One of the best presentation I have seen. Thank you for such a good explanation of LDA.
You're most welcome Abhishek. Happy to hear that you enjoyed it :)
Thank you for that beautiful presentation! I learnt a lot from it and enjoyed it immensely
Phenomenal, sharing with some colleagues! This is absolutely fantastic, I really appreciate this. The audio alone would have helped me a ton, and this is visually beautiful as well!
Thanks Leslie, very happy to know that you enjoyed it! :)
I got inspired to use gensim after watching this video. Very well explained. Thanks for uploading.
Thanks for the comment! Glad to hear you found it useful. Gensim is great! Made my life so much easier when i was a Data Scientist working with Natural Language Processing. They also have some cool stuff on Deep Learning! :)
Do you have any idea about how to train domain specific model like finance or restaurant? I think we can use wikipedia data but I have no idea on how to parse domain specific data from wiki dump.
Beautiful tutorial on LDA. Thanks a lot. Please create more video tutorials like this.
Thanks Dillip! Happy you liked it!
I think my next video will be about making Resumes, but we'll see :)
You have done a fantastic job with this explanation, and with the video production. Thank you!
Thanks Chris! Glad you liked it! Also, thanks for the email. I'll get back to you during this weekend!
Excellent Job ! On my top 5 of UA-cam presentations so far ! So clear and detailed at the same time. I deeply understand LDA with your explanations and the process of using it for the purpose of document similarity computation task. And I agree, the animations and visuals are perfect :-) Great Job again (I know, I've already said it)
Thank you Amelie. Super happy to know that you liked it :) the whole thing was done in Keynote (since I'm working on a Mac) and if you have any detailed questions on how I did this or that, please let me know. will be happy to help
@@andriusknispelis8627 i'll do that! Thanks a lot again 👍🏽
I usually never comment on UA-cam videos, but this was awesome. Thanks a ton man. Keep up the good work.
Thank you Utkarsh! Very happy you liked it :)
@@andriusknispelis8627 I totally approve Utkarsh's answer. I was simply amazed by your presentation. Thank you for sharing your inspiring thoughts.
Thanks so much, Arujinho. If there's any part of the presentation that I can break into "how exactly I did this or that step by step" please let me know. Will more than happy to share :)
@@andriusknispelis8627 I consider your presentation near perfect and more in detail about the process of LDA is probably left to the auditor to inform himself. This leeds me to a question: Which source do you recommend do dive a little bit deeper into the mathematical background of the process - especially considering a researcher that is rather "mediocre" in probabilstics ;-)
When it comes to LDA I'd definitely recommend to watch/read anything from David M. Blei (the author himself)
One of his video lectures (1 hour 20 min long)
ua-cam.com/video/DDq3OVp9dNA/v-deo.html
When I studied it I haven't watched the videos myself, I just read articles. Here's one I can recommend from him:
www.cs.columbia.edu/~blei/papers/Blei2012.pdf
I also like this one very much, also I think the title is spot on accurate :)
Reading Tea Leaves: How Humans Interpret Topic Models
pdfs.semanticscholar.org/3a99/da22b1658695d95a764169e030cc40e2fb95.pdf
The explanation is so straightforward and intuitive! Thank you!
Thanks Li! Happy to hear you liked it :)
Very clear explanation and beautiful voice. Will watch it all over again
THANK YOU so much - glad to know you enjoyed it!
Wow! Thank you, this was so immensely helpful, with a real-life example used to illustrate the whole way. Even the things that were highly technical were grounded in the ASDA magazine example and it really helped my comprehension, because everything I'd read before this was very abstract. Keep the videos coming!!
Thank you! I have to admit that reason I chose ASDA magazine was because I like their covers :). And the reason I went for that particular issue is because of the green color, since it went well with the rest of the slides :)
Thank you for the time and effort in making this video it is amazing (well designed, simplified, and most importantly informative).
Awesome Job Andrius. Great explanation and visuals. Very professional as well as effective. I've been using an implementation of LDA in R for a project, but had problems understanding the theory - this really helped.
Thanks Wes! Glad to hear you liked it and even happier that you found it useful. Makes it all worthwhile :) It was definitely my most used machine learning tool while i was a data scientist at issuu.
one of the best presentation for beginner like me
Amazing intro to LDA, thank you very much
brilliant work mate, very detailed, yet not too complicated
Thanks buddy! happy to hear you enjoyed it! :)
Dude, such a great presentation. Thank you very much for this superb explanation! :)
such a brilliant topic I ever see in my life. thanks indeed.
You're most welcome Omar! Happyto know you liked it :)
Dear Andrius Knispelis
I would be grateful if you can share with me the source code for this implementation. or atleast help me to access the source code in gensim.
Regards,
Omar
Very intuitive. Your explanation and illustration. Thanks for sharing your knowledge! 😃
Thanks Sheetal! Happy to hear you liked it! :D
Great presentation! Especially the part with the DNA.
Thanks! :) Glad you liked it. The idea of comparing to DNA came from something being similar enough to comparable, while still being unique.
Thanks for such easy, simple and clear explanation
Great presentation. Extremely well paced with a great overview
A great insight towards the working of LDA and its application
Thanks Nihar! I thought it I was gonna make one video then it had to be about the LDA. :)
Awesome video, brilliant explanation and loved the visuals. All the best.
Thanks so much for making this! Wonderful video! Will share this with my classmates!
Thanks and please do. I really appreciate it :)
High Standard Presentation, Thank you very much
Best online!
Clearly explained and beautifully presented!
Nice work! I can actually explain how this works the business folks now! Thank you.
The presentation is really well done. Congratulations and thank you.
Found this trying to learn about linear discriminant analysis, stayed because it was a week put together presentation on an interesting topic
Wow thanks a lot! Very clear explanation! Please keep posting videos on similar topics!
Best explanation of LDA! Thank you so much!
Great explanation and really appealing slides.
Thank you very much!
Thanks! Happy to hear you enjoyed it :)
The video was really helpful. Thanks for making it.
Amazing Video. Explanation is great. Visuals made it very easy to understand.
Thank you Sunil! Glad to hear you liked it. If you have any questions on the material or the visuals just let me know.
Compact, crisp and strong narrative video presentation...I watched it only 2 times and understand the process thoroughly....1 question, just to get your insight..Is LDA can be combined with systematic literature review protocol (SLR) and the produced model in LDA is similar with structural equation modeling (SEM) model?
Nice explanation. It gives me an idea how to present the work.
Thanks Yerik. Happy to hear you enjoyed it! What work are you peresenting, if i may ask..
My defense, which is about feature engineering on text data. Topic modeling is not my focus but you give me some ideas how to show your detailed work.
Excellent Video Andrius !
Thank you Madhavan! Happy to hear you liked it! More to come in 2017 :)
Wow. Fantastic explanation. Thanks so much.
This helped me understand LDA, thanks.
Glad to hear you liked it ! Cheers! :)
This is by far the most clear explanation on LDA. Which tool did you use to create this presentation? Beautiful!
Thank you Junqi! Glad to know you liked it. It's all made in Keynote (since i am using a mac).
Excellent video man. Just what I needed to brush up. I like the 2 in the binary code at 2:06 by the way.
THANK you man! :)
Hehe, that 2 was NOT a coincidense. It was really counting on somebody noticing it. And it made my day when i read your comment :) Cheers!
This presentation is excellent. Thanks.
Excellent and creative use of the algorithm!
Thanks Sam! Glad to know you enjoyed it! I'd argue that perhaps the creative part of it was not using LDA topics as some sort of final classification, but instead as a fingerprint and then clustering those fingerprints separately outside of LDA.
Thanks for sharing the video. very informative. Question? LDA returns me a score for each topic when I infer a new document? how do you convert that score into the fingerprint?
hi Saurabh, that probability distribution that LDA gives you IS a "fingerprint", or "DNA", or "whatever_other_term_you_can_come_up_with". Basically, it's just another way of saying that it gives us something unique for each document.
Hi Andrius, thank you so much. This is a really amazing video, it's very well explained and helped me with my current project. Please make more videos!! Thanks.
Hey Tony! Glad i could help out :)
What's your project about?...
I have been working on social media analysis.
Excellent presentation. A very impressive project. Thank a lot.
Well laid out and clear presentation. Thanks!
Thanks Jobi! Happy you liked it :)
This one was super helpful, thank you very much!
It was one of the best presentations I came across. A good video.
Andrius - Can you guide, on how you have created the dashboard showing words left, not in lda, in lda, unique.
Also, what are the graphs at the top right of the presentation?
THANKS! I really appreciate it :) Happy you liked it.
OK, so in the top right there are three graphs there (yellow, green and blue).
Yellow - the similarity (JSD from 0 to 1), so i can see both the size of the neighbourhood and how quickly it dissolves into the rest of the documents (the slope of the curve). If it's a newspaper with many themes, it dissolves slowly, if it's a very concrete niche magazine - that curve is much more sharp.
Green and Blue are both showing the same documents as in yellow one in the same order, but Y axis is showing the number of words in there. Green is total words (same as in "words left" on a grey barchart a bit to the left), Blue is unique words. I wanted to see how the LDA similarity relates to the number of words in a document.
All three graphs only show the top 300 neighbours.
First barchart, the one with all the colors, is simply taking all words from a document and matching them with a buch of lists. I had lists with city names, country names, people names, and so on. I have removed all of those words, only left the ones marked in white color there (in the top)
The second barchart starts where the first one ends - first number is how many words are still left in the document after removing ones that triggered the stoplists. Then i broke that first number in two parts: a) words that were not in LDA model, and words that were. Then final number is how many of those were unique. So i know it's not just a several words being repeated a lot.
Oh, and all those graphs were created using Python with Matplotlib package, and producing an HTML file. I was running hundrends of these tests, each generating an HTML file with a bunch of magazines, so i can easily browse through and see how it looks :)
it was this easy to understand...you made it easy...thanx!!
Thanks!! Happy to hear you liked it :)
This is incredibly clear and helpful. Thank you!
thanks Richard! Making it clear was indeed one of my main goals here.
I made this video right when i was making a shift from Data Science into Product Management. And I used it as one of my "portfolio items" since I wanted to show that i can take something technical and explain it. That was one of the reasons how this video came to be at that particular time.
Happy to hear you liked it :)
Best LDA video ever! Thanks a lot
Thanks Yazid :)
Hey great presentation Andrius! Just curious are you planning to make more videos of different NLP models or ML models in general? The ideas are well explained with sufficient details and the way you organized and presented is awesome; it would be great if you could make more of these and share your wisdom with the rest!
Hi Shum, happy to hear that you enjoyed it. When I was a Data Scientist back at issuu, LDA was the thing I worked on basically daily, so I got to know it pretty well. Which is why I made the video on it. Sadly, I can not say that I'm familiar with other NLP models in the same way... So if I make more videos it will probably be about something else :). I'm thinking doing a little series on how to build your CV and something more on visualisation and/or presentations.
Ačiū už šį video, jis nuostabus!
Ner uz ka, Tautvydai! Smagu, kad patiko :) pirmas lietuviskas komentaras, hehe
best video to understand LDA topic model..
Awesome lesson, and perfect presentation.
Thanks Firas! Glad to hear you enjoyed it :)
Brilliant presentation. Much appreciated.