Detecting Power Laws in Real-world Data | w/ Python Code
Вставка
- Опубліковано 6 чер 2024
- This is the 2nd video in a series about Power Laws and Fat Tails. In this video, I describe how to objectively detect Power Laws from real-world data and share a concrete example with social media data.
📹 Series Intro: • Pareto, Power Laws, an...
📹 Fat Tails: • 4 Ways to Measure Fat ...
📰 Read more: medium.com/towards-data-scien...
💻 GitHub Repo: github.com/ShawhinT/UA-cam-B...
References
[1] arXiv:0706.1062 [physics.data-an]
[2] arXiv:2001.10488 [stat.OT]
[3] en.wikipedia.org/wiki/Likelih...
[4] en.wikipedia.org/wiki/Pareto_...
--
Book a call: calendly.com/shawhintalebi
Homepage: shawhintalebi.com/
Socials
/ shawhin
/ shawhintalebi
/ shawhint
/ shawhintalebi
The Data Entrepreneurs
🎥 UA-cam: / @thedataentrepreneurs
👉 Discord: / discord
📰 Medium: / the-data
📅 Events: lu.ma/tde
🗞️ Newsletter: the-data-entrepreneurs.ck.pag...
Support ❤️
www.buymeacoffee.com/shawhint
Intro - 0:00
Power Laws Break STAT 101 - 0:59
Log-Log Approach - 2:21
Maximum Likelihood Approach - 4:54
Example Code: Artificial Data - 8:11
Example Code: Real-world Social Media Data - 16:16
What's Next? - 22:29
📹Series Intro: ua-cam.com/video/Wcqt49dXtm8/v-deo.html
📰Read more: medium.com/towards-data-science/detecting-power-laws-in-real-world-data-with-python-b464190fade6?sk=07960e2c880b7f6f5ac577e6beb843a3
💻GitHub Repo: github.com/ShawhinT/UA-cam-Blog/tree/main/power-laws/2-detecting-powerlaws
It was fascinating 👍But the sound was a bit low.
Thanks 😁 I’ll improve that on the next one
very interesting as usual 👍 wondering how does one handle missing values in such cases?
Good question. It depends on on what a "missing value" is here. Is it missing because of an issue in the data collection process or because the event is so rare. In the former case, you'd need to correct the issue on a case by case basis. In the latter case, you'd need to collect more data.
Hope that helps!
Hi Shaw, just trying to get my head around PL distributions, let alone Python. However a question re Chapt 6 Example Code: Fitting PL to SMD, viz Medium Followers. Wouldn't Exponential be a better fit given R>3.9/p>.7, against lognorm (ect) R>-12.7/p>.007
Good question! In this context, smaller p means better fit.