Hello Emma, thanks for your sharing, may I ask for a question, for weighted combination of, say 3 metric, how do you decide the weight among 3 metrics? For example, the engagement of FB newsfeed, the number of likes, comments and shares, there are 3 main metrics to measuring the FB newsfeed engagement, compare to like, share and comment are more meaningful to measure the interaction on FB, but how would you decide the weight among those 3 metrics in a scientific way? Really thanks for your answer.
Emma - watching your videos helped me land a data scientist position at a FAANG company. So grateful for your knowledge and your ability to share it with a wide audience :)
Hi Emma, thanks for the super clear and structured content! I am still a bit confused about what an "attributable" metric means. Could you add more clarity here? Thanks!
Hi Emma, for the sample ratio mismatch, shouldn't we use z-test rather than t-test? I read that the variance p(1-p) and mean p are not independent for proportion comparison, so it will follow a z-distribution.
When it comes to hypothesis tests of proportions z-test should be used. However in practice t-tests are often used and they given results very similar so it's ok (timestamp 6:40): ua-cam.com/video/IY7y-t30UJc/v-deo.html
Thanks, Emma for your contributions to our DS community! I have been enjoying your content so much so I even clicked on the like button before I watch any of your new videos :D. Thank you very much!
i love your channel! you should create amazon referral links for the book you mentioned in your video. that way you can get referral revenue when your followers buy that book based on your recommendation. i want to read that book now
We actually use a 3rd category of guardrail metrics. Program metrics. To ensure overall quality of the experimentation program. One of the guardrails we have is ratio of iterative to exploratory tests.
Fantastic observation! When metrics go a bit wild we can experience strange outcomes for all kinds of reasons worth talking about. This has given me a great idea for more content about how to avoid analysis paralysis! Thanks for sharing your comment with me!
6:13 for the sample ratio mismatch check, I think we should only apply Chi-square test, which is to compare a proportion with an expected value. Can you help clarify how to use a T-test for checking SRM? Thanks
The chi squared goodness of fit test and the one sided one sample t-test of proportions are the same exact thing. ua-cam.com/video/-Vssir6yUNQ/v-deo.htmlsi=EpNOH7LCnFC2fNmF
It's because the chi squared goodness of fit test is the same as the one sided one sample z-test of proportions: ua-cam.com/video/-Vssir6yUNQ/v-deo.htmlsi=EpNOH7LCnFC2fNmF When it comes to hypothesis tests of proportions z-test should be used. However in practice t-tests are often used and they given results very similar so it's ok (timestamp 6:40): ua-cam.com/video/IY7y-t30UJc/v-deo.html
Typo correction:
Thanks Squiderify, at 10:38 "metics" should be "metrics".
Hello Emma, thanks for your sharing, may I ask for a question, for weighted combination of, say 3 metric, how do you decide the weight among 3 metrics? For example, the engagement of FB newsfeed, the number of likes, comments and shares, there are 3 main metrics to measuring the FB newsfeed engagement, compare to like, share and comment are more meaningful to measure the interaction on FB, but how would you decide the weight among those 3 metrics in a scientific way? Really thanks for your answer.
这节说的真好，微软AnE说的也不过如此。口才好，写书的Rony若是听了也要点赞。
Emma - watching your videos helped me land a data scientist position at a FAANG company. So grateful for your knowledge and your ability to share it with a wide audience :)
Really intuitive videos, Thanks! Keep up with the good content!!
Good explanation of metrics 😊 thanks for sharing 👌 keep going 👍
Hi Emma, thanks for the super clear and structured content! I am still a bit confused about what an "attributable" metric means. Could you add more clarity here? Thanks!
哈哈哈，你的内容真好，请问可以在Github或者medium上同步吗，给点文件啥的
Hi Emma, for the sample ratio mismatch, shouldn't we use z-test rather than t-test? I read that the variance p(1-p) and mean p are not independent for proportion comparison, so it will follow a z-distribution.
When it comes to hypothesis tests of proportions z-test should be used. However in practice t-tests are often used and they given results very similar so it's ok (timestamp 6:40): ua-cam.com/video/IY7y-t30UJc/v-deo.html
Thanks, Emma for your contributions to our DS community! I have been enjoying your content so much so I even clicked on the like button before I watch any of your new videos :D. Thank you very much!
Glad my videos are helpful!
数科秘籍Emma Ding，interview rat 大救星
This channel is gonna be a great technical resource for professionals. Thanks 👍
i love your channel! you should create amazon referral links for the book you mentioned in your video. that way you can get referral revenue when your followers buy that book based on your recommendation. i want to read that book now
Best videos
We actually use a 3rd category of guardrail metrics. Program metrics. To ensure overall quality of the experimentation program. One of the guardrails we have is ratio of iterative to exploratory tests.
给Emma点赞！！最近面试Product Analytics 多亏看了你的视频！！讲的又好又清晰！！
因为自己没有Product经验，所以面试还挺费劲儿哈哈~ 希望自己可以早日上岸！再次谢谢Emma的视频
I want to cry, I understand now, how much effort goes into videos like these! I in love with you!!
Hi Emma, could you dive deeper into how to answer the question “If one metric goes up, and the other goes down, what should we do?”
Amazing content and often asked in technical interview.
Keep it up Emma 🙌
so useful and helpful!!! I learned a lot from your video! thanks so much
You explain things so well, very helpful for those of us breaking into the data science world. Thank you Emma
Great source of information, and clear explanation for tough topics
good explanation, please keep up the good work!
hi Emma, thanks for your sharing, could you please share more detail about “If one metric goes up, and the other goes down, what should we do?” thanks
Fantastic observation! When metrics go a bit wild we can experience strange outcomes for all kinds of reasons worth talking about. This has given me a great idea for more content about how to avoid analysis paralysis! Thanks for sharing your comment with me!
Can u shows its implementation so that it easy to remember.
6:13 for the sample ratio mismatch check, I think we should only apply Chi-square test, which is to compare a proportion with an expected value. Can you help clarify how to use a T-test for checking SRM? Thanks
The chi squared goodness of fit test and the one sided one sample t-test of proportions are the same exact thing.
ua-cam.com/video/-Vssir6yUNQ/v-deo.htmlsi=EpNOH7LCnFC2fNmF
It's because the chi squared goodness of fit test is the same as the one sided one sample z-test of proportions: ua-cam.com/video/-Vssir6yUNQ/v-deo.htmlsi=EpNOH7LCnFC2fNmF
When it comes to hypothesis tests of proportions z-test should be used. However in practice t-tests are often used and they given results very similar so it's ok (timestamp 6:40): ua-cam.com/video/IY7y-t30UJc/v-deo.html
Thank you for sharing
Thanks for sharing
At 10:38 there is a typo: "metics" instead of "metrics". Amazing content though!
Thanks for spotting the typo!