Network of The Witcher | Relationship Extraction & Network Analysis with Spacy & NetworkX
Вставка
- Опубліковано 3 лип 2024
- 🔑 GitHub repo for this project: github.com/thu-vu92/the_witch...
📹 First episode of this series: • Python Web-scraping wi...
🙇♀️ Graph Analytics for Big Data(Coursera): imp.i384100.net/oeWP1O
🤓 Join my Discord server: / discord
Hey data nerds 👋, in this video we made an attempt to extract relationships between The Witcher's characters from the books. We also did some graph analyses (centrality measures and community detection) and visualization using NetworkX and Pyvis. The analyses are still quite preliminary and I might have glossed over some details, but hopefully, this is a bit interesting and you learned something from it 🤓. This is the second project in the Witcher project series where we will analyze the relationships between characters. The next episode of this series will be all about network graph visualization 😁. I hope you enjoyed this video and any questions and thoughts are welcome! Thank you for watching 🙌🏽
👩🏻💻 COURSES & RESOURCES
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
📖 Google Advanced Data Analytics Certificate 👉 imp.i384100.net/anK9zZ
📖 Google Data Analytics Certificate 👉 imp.i384100.net/15v9y6
📖 Learn SQL Basics for Data Science Specialization 👉 imp.i384100.net/AovPnJ
📖 Excel Skills for Business 👉 coursera.pxf.io/doPaoy
📖 Machine Learning Specialization 👉 imp.i384100.net/RyjykN
📖 Data Visualization with Tableau Specialization 👉imp.i384100.net/n15XWR
📖 Deep Learning Specialization 👉 imp.i384100.net/zavBA0
📖 Mathematics for Machine Learning and Data Science Specialization 👉 imp.i384100.net/LXK0gj
📖 Applied Data Science with Python 👉 imp.i384100.net/gbxOqv
🙋🏻♀️ LET'S CONNECT!
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🤓 Join my Discord server: / discord
📩 Newsletter: thu-vu.ck.page/profile
✍ Medium: / vuthihienthu.ueb
🔗 All links: linktr.ee/thuvuanalytics
🔑 TIMESTAMPS
================================
0:00 - Intro & objectives
0:47 - Installing packages & project setup
2:07 - Plan for the project 🧭
3:34 - Get data & process data
5:13 - Read in character names
7:06 - Get character list per sentence
9:47 - Create relationship df
14:22 - Network graph visualization & analysis
18:40 - Centrality measures
20:26 - Community detection
21:54 - Characters' importance over time
25:25 - Where to learn graph analytics
26:18 - Some announcement 😉
================================
As a member of the Amazon and Coursera Affiliate Programs, I earn a commission from qualifying purchases on some of the links above. It costs you nothing but helps me with content creation.
#networkanalysis #Datanerd #DataAnalysis #datascience #ThuVu #dataanalytics
Join my discord server discord.gg/SK7ZC5XhcS
The video many of you were looking forward to! 🤓 It was really fun to make. I hope you enjoyed it and found it useful for your own projects! 😊
Wonderful. TQVM!!!
says the invite has expired?
@@danielq1985 Hey Daniel, I forgot to change the expiry date of the invite, sorry! 🤦🏻♀️ I've changed the link in the comment above!
@@Thuvu5 no worries! Thanks for updating it and replying so quickly! This is an amazing thing you are doing btw, really. This video is very timely. I am doing a text analysis on a completely different subject but this tutorial will 100 percent help me. So thanks!
Can you pin this or better add this to the description?
We NEED more of this kind of projects!! I love this series so much... So please, we need the finale of this project...
Heck yeah! Loving where you are taking this video series, Thu!!! 🙌🏼🙌🏼
Thank you so much, Luke! 🤗 It took me forever to put all parts together, I'm so glad to finally put it out.
Graph algorithms are quite useful as well as beautiful in their own right.
I’m glad there are some people using them.
More importantly, use them with a fun-to-do use case.
Thanks for your effort.
Omg this is the best format for learning this topic. love
Love the flow of this video. Amazing step by step. Thanks Vu.
Hey Chris, thanks so much for your kind words! Glad you enjoyed it 🤗
Was completely captivating from start to finish
Hey Thu Vu, this stuff is really very cool!! Watching what amazing things we can do using Data Science excites me. Thanks for sharing it with us.😀
this was incredible!
following thru took me 3 hours, i didn't know data science could be this fun, please make more!!
Amazing that you followed through the project, Kanaan!! 🙌🎉 I’m so glad you had fun doing it 😀
Bravo! Absolutely a great job, keep going. Best lucks!
Very cool visualization and awesome topic!
Thu, I absolutely love this video series and lost count of how many times I have watched it for tips & tricks. Thank you 🙏 can’t wait for the computation.
Thank you David! 🙌 This is so amazing 🤩
I was looking forward to this video. I'm loving the series of video. I can't wait to put this into practice.
So glad to hear Vinicius! 🙌🏽
You are really providing amazing content.
Wow Thu, just found your channel and I'm hooked! Thank you!!
I really love your work! Good job there, very inspiring
Wow, the best explanation of the DS project on YT I have seen! 👏🏻Thank you for educating us!
Aw thank you! Really appreciated your kind words 🙌❤️
More awesomeness like this please, data visualisation is my favourite 😀
Thank you very much for these kinds of videos. I am currently working on 2 projects that require these two techniques, and this video has been very helpful to me. Thank you so much!!
Love this!! I love the format you use for explaining these concepts. Super helpful :)
Aw thank you for this, Kyle! 🤗🙌
Very interesting content. Thank you! 🙏🏻
Glad to see the end product! Nicely executed
Hey Shivam, thanks for your comment and your help with ideation! 🙌🏽🤓
I thought I love your videos, but now you made a video about The Witcher's universe?
Aw...! I didn't think I could love them even more...
Aw thanks for this haha 😇. I just thought I could do something to fill my time until The Witcher season 3 😜
And I love you even more!!!! MORE please!!
Very well done!!
Thank you thank you. I love learning by doing projects so this is perfect. Thank you! 😀
this video was sooo helpful!!! thank you.
I love your posts very much!
Really well explained code, it's rare to find such a good description of an algorithm. Thank you!
Thank you Loren!! 🙌🤓
This is great!...keep em coming
Thank you Juan! More to come! 🙌🏽
Such a great video. Thank you so much for sharing this work and insights.
Excellent video. Thank you
Love the editing of this video, Great Video Thu
Thank you! It took me some time to edit so I really appreciated that ❤️
this is so... COOL! I like your tutorial style, it's right to the point.
Thank you so much Daniel! 🙌
Amazing video and amazing channel ! Thanks a lot !
This is so fantastic. So so cool. 👏🏻👏🏻👏🏻
Thank you so much Andrew!! So glad you enjoyed it! 😄
This video is awesome! Great job Thu!!!
Hey Smitha! Thank you so much for watching my video and your kind words❤️. I just came across your channel recently when i started making the ML course review 🙈. You’re doing great with your content, keep it up! 🙌🙌
Omg, amazing video!!! I am starting with graph theory and visualization and this video has very important information for my project. Special thanks for the hints on extra material. You teach very well! Keep up the excellent work😊
Wonderful! Good luck with your project 🤗
My partner will love this! Thank you :)
Great channel!!! I'm really enjoying your content.
Thank you so much Troy! I’m so glad to hear you liked the content 🙌
Great video, enjoyed it very much. And great choice of subjects, turn what had the potential of being "a very boring lesson/tutorial" into an exciting adventure that had me clinging to my chair thinking "what are we going to discover next?"
great video. thank you!
Thank you very much!
The use of lambdas is impressive.
Seriously, Data Science is just mind blowing! Specially when you consider all the real-world applications for those resources.
Totally agreed!!
Love the haircut!!! Great video will try this for my learning project!!!
Aw thanks Kaustav! 😇 Have fun with your project!
This was so entertaining to watch, never knew you could do that with nlp, thanks for this amazing video!!! i'll try latter with different fantasy books.
Hope you have an amazing week!!!
Aww thank you so much! You too, have a great week! 🙌
First! Amazing video, Vu!!!
Yaaay first comment! 🙌🏽😍
Your content is great! I found your video because I was lost in my data analysis class when it came to the module of Network Analysis. I'm going to recommend you to all my classmates. Thank you for putting so much thought in this!
Thank you so much for your support! I’m glad the video was helpful 🤗
same experience here 💀
Thanks. Your videos are really helpful
Glad to hear Jacky! 🙌🏽
Phenomenal!
Love it!
Interesting, we think a very similar. I got a patent for the approach you described. I should include you in the inventors list for next patent.
Very cool !!
Wow, The topic of this video is that good. I hope you can create many good videos. 😍😍😍
really cool! do more social network analysis
What a great video!!
Keep up the good work.👍 Btw this haircut looks great on you😬
Haha thank you so much Harshal! Thanks for your help with the visuals ❤️
I LOVE the copy pasting thing you do. No time wasted typing. Thank you! You're awesome.
Thank you so much for this Henock 🙌!
@@Thuvu5 Would you mind sharing how you are doing this style of copy pasting Thu Vu? It looks very clean and is great for these videos. Do you have some kind of tool for this?
@@alexvv2053 No, it's proprietary 😏. Jk, I'll share it in another video 😂
keep up your good work :)
i like the way you explain, keep it girl
Aw thank you so much Bagus! 🤗
Thanks!
cool quá em 😍
A good intensive lecture , hope you spilt it to absorb more valuable insights .
Thanks
hey Ahmed! Thanks for your comment. Yeah.. It's probably better to split the video 🙈
I smashed the subscribe button as soon as I saw the thumb nail.
Yaay, thank you! 😍
Hi I loved this video and lot of it is familier with my current project on static graphs on player interactions playing pc/console games. However, my current research takes it a noch higher on the same lines but with dynamic graphs. Currently I amtrying to implement the same with networkX + dynetX, with a keen eye at pathpyG as well. I would love to see a future video on dynamic graphs!
amazing
Thank you very much Madam.
Coming from the life sciences, I’ve only used cytoscape for network related stuff. It’s rest api isn’t very good, so it’s a bit difficult to automate, but it has more features than networkx imo. Great tutorial. I did something similar in a side project a while ago for Japanese novels. Was a lot of fun.
Oh I didn’t know cytoscape but sounds like a nice tool. Interesting that you did the same project for Japanese novels 😃🙌
a science is a science, what are you talking about karen?
This project is unbelievably fucking cool
Thank you Thomas! So glad you liked it 🙌🏽
nyce approach
Omg awesome!!!!
Thank you Justin 🙌😍
Really proud of you. Cuz you are a vietnamese
Wonderful. I will try with LOTR & GOT.
Oh that sounds awesome!! 🙌🏽😍
awsome, good material, and let me thinking were can i used it for my life!
That sounds great!! Thanks for commenting Agustin! 🙌
Love your videos. Is it possible to make a video regarding to the data cleaning, such as how to treat outliers, etc? thanks
This is such a cool project! I had not heard of Spacy or NtworkX before.
I have only seen the TV series, so please don't hate me!
Oh oh of course I hate you for this 😜. Thanks for watching Cereal Port! 👋
I'll watch another time.
Love your content. It's easy to follow. Can't help but feel sad for the hair though
Thank you! Haha it will grow back in no time 😂
Amazing project and presentation, thank you!
Thank you so much, Chris! 🙌
Cool video
Thanks Ethan 🤓
Thanks Thu Vu, really enjoy your informative videos! Would you be able to do a video on how you document and organise your data science projects as well? E.g. there was a snippet of how your created a separate python file in the utils folder to store your functions. It would be really helpful to see in greater detail how the whole project folder was organised. Thanks again!
Hi Bao! Thank you for your watching!! You can find the code structure of the project in the Git repo (in the vid description)
You are so smart
Oh my Goood, those animations in the intro 🥺🥺😭😭
Turorial please 🙏🏾🥺
This was great, when do you plan to do the 3D network graph visualization video though??
Please make a tutorial about association analysis!!
Great video. Loving your content!!
I've a question, how to display the graph into panel dashboard please?
You are awesome
Haha I’m trying! Thank you Mustafa 🙌
Great vid! Very interesting! Btw nice hair Thu Vu! Looking more cute haha
Aw thank you! Glad you liked the vid. Maybe this hair is too cute? 😂
@@Thuvu5 haha yeah. Btw can you help me? Can you create content about a simple data analysis using R? There's not enough good content about it here on youtube. Also to give people out there some idea about R. Thanks Thu Vu! You're the best for me! Numba wan fan from ph!
@@mischievousachiever7896 Hey, thanks for suggestions. I have some earlier videos about data analysis in R on my channel, please feel free to check them out. Hope it helps! Maybe i should create some more haha
@@Thuvu5 yeah, I think that would be a great idea. And just let me know whenever you need some suggestions about contents my friend. 😊 Hoping we can stay in touch in discord
@@mischievousachiever7896 For sure! There's a channel on Discord for new video ideas. Please feel free to post your ideas there! 🙌🏽 Thanks a lot for your help :)
11:21 char_unique another way to perform this would be char_unique = list(set(char_list)) # create a set of the entries and then convert it back to a list which ensures uniqueness
16:16 for the lazy ones you can run terminal commands within a jupyter cell with exclamation mark. e.g. !pip3 install scipy
Great Method!, one question: Which cam model do you recorded this video? , Thanks!
This is an awesome video. Thank You so much for this. Pls can you recommend books on Social Network analysis for me. I want to get started.
Hey Abimbola, thank you for your comment, I really appreciate it! I mostly learned through projects and haven't actually read Network analysis books, but this one seems very good rb.gy/ahjqgu
Hey @thuvudataanalytics you may want to re-check home of your play ▶ lists 📃 don't have accidental 3rd party entries, ie the DSci Playlist has some yoga 🧘🏻♂️ & other unrelated posts. FYI.
PS awesome tutorials!
Is the next video on graph analytics out?
Hey! Great video, thanks a lot! Awesome to see these libraries applied to an interesting problem in a cool way!
I would just have one question: what is the value of using NER and spacey in this case? If I understand correctly, you already had all the entities (characters) so perhaps the tokenisation of sentences so that you can calculate the rolling window? At the same time, wouldn't it be easier to make an assumption about the avg wordcount of 5 sentences, scan the document for your characters and their position, calculate their relative position and get the weights?
I'm curious if there is something I missed (feel free to point out if what I said doesn't make sense)
🙂
Hey Balazs, very good question! 😄 I was thinking the same when I was making this project. But in the end I think using NER will make it more accurate in identifying the entities. Because sometimes if we only match words by words we might accidentally identify non-entity words as characters, just because they have the same spelling. But I can imagine the method you described could work as well.! It may worth double-checking the output if it's working as expected.
@@Thuvu5 Thank you for the reply! Will share my results if I have the time to code this up
I wonder how someone could come up with a way to parse Ciri vs. the false Ciri from the text.
That’s a very interesting question! 🤔 I actually don’t know how to distinguish the fake Ciri. I was really mad at her at the end of season 2 as she killed so many witchers 🤦♀️
Honestly you'd have to use context clues and use outside information to know. I wonder if they'll even put her in the show.
The books are a more elegant story than the show 🫣
Maybe I misunderstood something but if [A, B, C, D] appear as characters in the window, shouldn't you create the relations A->B, A->C, A->D, B->C, B->D and C->D? In you example there are fewer pairs.
Hi Peter, no it's actually A->B, B->C, C->D. But you could also argue a different way to define the relationship :)
Hi! just wondering if we could do the same thing with gephi?
12:13 would i make sense to permutate all the characters in the given window? If we have A, B, C, why wouldn't you want to have a relationship between A -> C
if you would do that you could use
from itertools import
list(permutations(char_unique, 2))
Hi, your video was awesome, but i have question to the pyvis graphics. I have the problem that the nodes moving extremely fast and i cant fix that😢 can you help me? I use exactly the same code only another dataset.
Why was the whole NER (NLP) part necessary, if the character names were already identified via scraping? Wouldn't it suffice to just loop over the tokens of the book and select for each sentence the token if it is found in the list of characters?