Hi Emma, thank you for the helpful video! I found on the side when you were introducing Experiment Design, the bullet points number for "size of an experiment" and "how long to run an experiment" are both 2.
Hi Emma! Great video as always. You mention that in experiment design you use seasonality, day of week, primacy and novelty effect to determine how long to run the test for. But in your A/B testing interview video you only mention sample size. I must’ve missed something but are you able to explain why there’s that difference? Thank you!
Do you plan to make the slides available? It would be extremely useful. If so, you could also put a watermark on it because I bet it would be shared very often.
Hey Emma, thanks for uploading this. It's very helpful. And I've a question here: what's the difference between sample size in experiments and the number of randomization units? Thanks in advance.
Love these videos on AB Testing! Any chance we can get a video on methods of experimenting when AB-Testing is not possible? (i.e. Synthetic Control methods, switchback experiments, etc.)
Hi Emma, I found a lot of case of A/B testing if about the website users. What is there is a national retail chain company that has 1000 stores around the country. And they want to test if a coupon can help increase sales. In this case, what is the randomization units here? the stores or the customer? Does A/B testing still apply in this case?
I agree with this suggestion. Emma, you have a lot of useful videos in the playlist but if you could sort in the order you think is the logical sequence of learning A/B testing, that'll make the playlist even more helpful. Right not this video is at the bottom of the playlist but it seems you think this is the starting point of learning instead.
This is really helpful!! Not sure how easy this is to do but I was wondering if you would be able to make transcripts available for the video. (I personally have a learning disability so it really helpful for me to have a transcript to accompany a video. Thought this might be also helpful for people where english isn't their first language too)
Emma, Thank you for this helpful video! Could you please help me understand the difference between "have enough randomization units" and "size of an exp".
Thanks for the great video! How would you design an A/B test for a social feature on a social media site (e.g. facebook stories)? The users are interconnected with each other via posts, shares, likes, etc., rather than having their own isolated experience such as on e-commerce websites. If we randomly spilt users into 2 groups for the A/B test, users in the A group are going to be interacting with users in the B group. Could that bias the results?
Absolutely! The way interaction effect is overcome in networks is by using cluster sampling. LinkedIn uses the concept of ego networks to identify clusters and randomization is done at the cluster level. This will minimize spill over effect. Metrics will be calculated at the cluster level and averaged at treatment and control group level.
Hi, thank you so much for the great content! But I can not receive the "product case interview cheat sheet" even I registered my name and email through the link. How can I receive it? Thank you so much!!!
Thank you for your interest in our product case interview cheat sheet! We have resent the email. Feel free to reach out to info@datainterviewpro.com if you have any other questions.
HI emma. I have a question, so why don't people do sanity check before running the experiment? So they can save energy on running an unqualified ab testing.
Thanks for uploading this video, Emma. It' very helpful. I had a doubt regarding A/B/N testing and multivariate testing. I designed a full factorial experiment where I had 6 groups (factor A had 3 levels and factor B had 2 levels) without a control group. I checked that my residuals had constant variance and my data was normally distributed and so I was able to use Tukey's multiple comparisons. The result was that the best two groups (highest mean response indicates best in my use case) were not statistically different and even though it was practically significant as per an arbitrary self established metric, I would not recommend the winning group as it was not statistically significant (I believe this is the sanity check as the investment in winning group was worth it). I wanted to know if this problem falls under A/B/N testing or multivariate testing? Thanks in advance!
Hi Emma. Thank you so much for your video. Can you make a video on what to include in a resume to increase the chances of resume selection for DS interview?
谢谢Emma! 你的视频真的很有帮助～继续加油！
Emma guided me through my interview prep, and I got 3 DS offers, she had suggested this book to me, this book is awesome.
That's awesome news! Congrats on your offers!
do u recommend it ? does it covers all she explained ?
Hi Emma, thank you for the helpful video! I found on the side when you were introducing Experiment Design, the bullet points number for "size of an experiment" and "how long to run an experiment" are both 2.
谢谢分享！视频条理清晰！期待更多视频！
Hi Emma! Great video as always. You mention that in experiment design you use seasonality, day of week, primacy and novelty effect to determine how long to run the test for. But in your A/B testing interview video you only mention sample size. I must’ve missed something but are you able to explain why there’s that difference? Thank you!
Do you plan to make the slides available? It would be extremely useful. If so, you could also put a watermark on it because I bet it would be shared very often.
That's a good suggestion. However the "slides" are actually video editing, i.e. part of the video. I will keep your suggestion in mind. Thanks.
Hey Emma, thanks for uploading this. It's very helpful. And I've a question here: what's the difference between sample size in experiments and the number of randomization units? Thanks in advance.
Love these videos on AB Testing! Any chance we can get a video on methods of experimenting when AB-Testing is not possible? (i.e. Synthetic Control methods, switchback experiments, etc.)
Hi Eze, thank you for the feedback! I'll definitely work on creating more helpful content. Stay tuned! :)
Hi Emma, I found a lot of case of A/B testing if about the website users. What is there is a national retail chain company that has 1000 stores around the country. And they want to test if a coupon can help increase sales. In this case, what is the randomization units here? the stores or the customer? Does A/B testing still apply in this case?
Emma thanks for another amazing video! 希望能看到你出更多的product sense相关的视频:D
Feel like I have a lot to learn about ab testing. Can’t wait to see the next few videos!
Thanks Emma , for making these videos. Looking forward to the next part 👍
COULD I GIVE YOU A SUGGESTION? IT IS BETTER TO ARRANGE YOUR VIDEOS IN ORDER.
Hey Wei, this playlist may be helpful for you ua-cam.com/play/PLY1Fi4XflWSvgsaD9eXng6N5kxcMtcxGK.html :)
I agree with this suggestion. Emma, you have a lot of useful videos in the playlist but if you could sort in the order you think is the logical sequence of learning A/B testing, that'll make the playlist even more helpful. Right not this video is at the bottom of the playlist but it seems you think this is the starting point of learning instead.
This is really helpful!! Not sure how easy this is to do but I was wondering if you would be able to make transcripts available for the video. (I personally have a learning disability so it really helpful for me to have a transcript to accompany a video. Thought this might be also helpful for people where english isn't their first language too)
I would love to have more of this videos.. 😃
thanks for your videos! Plz keep updating these great contents and it is super helpful!
This video helps me in learning statistics and probability. Thanks Emma for making this video.
Emma, Thank you for this helpful video! Could you please help me understand the difference between "have enough randomization units" and "size of an exp".
Thanks for the great video! How would you design an A/B test for a social feature on a social media site (e.g. facebook stories)? The users are interconnected with each other via posts, shares, likes, etc., rather than having their own isolated experience such as on e-commerce websites. If we randomly spilt users into 2 groups for the A/B test, users in the A group are going to be interacting with users in the B group. Could that bias the results?
Absolutely! The way interaction effect is overcome in networks is by using cluster sampling. LinkedIn uses the concept of ego networks to identify clusters and randomization is done at the cluster level. This will minimize spill over effect. Metrics will be calculated at the cluster level and averaged at treatment and control group level.
Hi Emma, Any book you recommend to learn Root Cause Analysis or Failure Analysis ? Thank you
An experiment. I still dont get it. How can I estimate how long to run A/B testing?
@Data Interview Pro Please can you share all slides of A/B testing using google drive or any other application etc.
nice video, your blue sweater perfectly echoed with google color schema test!
Hi, thank you so much for the great content! But I can not receive the "product case interview cheat sheet" even I registered my name and email through the link. How can I receive it? Thank you so much!!!
Thank you for your interest in our product case interview cheat sheet! We have resent the email. Feel free to reach out to info@datainterviewpro.com if you have any other questions.
HI emma. I have a question, so why don't people do sanity check before running the experiment? So they can save energy on running an unqualified ab testing.
Hi! Good question, we should do it before and after.
Thanks for uploading this video, Emma. It' very helpful. I had a doubt regarding A/B/N testing and multivariate testing. I designed a full factorial experiment where I had 6 groups (factor A had 3 levels and factor B had 2 levels) without a control group. I checked that my residuals had constant variance and my data was normally distributed and so I was able to use Tukey's multiple comparisons. The result was that the best two groups (highest mean response indicates best in my use case) were not statistically different and even though it was practically significant as per an arbitrary self established metric, I would not recommend the winning group as it was not statistically significant (I believe this is the sanity check as the investment in winning group was worth it). I wanted to know if this problem falls under A/B/N testing or multivariate testing? Thanks in advance!
Thank you!
Hi Emma. Thank you so much for your video. Can you make a video on what to include in a resume to increase the chances of resume selection for DS interview?
Hey Manjari, I have a video on the exact topic you are asking, here you go ua-cam.com/video/xSTCmcxX-RI/v-deo.html!
thank you. See you soon!