I wish you had mentioned the feedback would be on another website at the beginning of the interview. Also, is this person who is being interviewed a real candidate or someone who is already working as a DS at Amazon (or somewhere else?)
He A/B tested it and found out that more peole will watch the video, and more people will move to paid subscription. Major dick move, but this part isn't measured and not important.
Although it would seem more straightforward at first blush, it wouldn't be a particularly favorable metric in this case. See, if a certain user makes several searches and/or purchases (which is inevitable, in practice), the underlying observations that contribute to the metric are correlated with one another, leadning to a violation of the t-test's assumption of observations' independence. Think of an extreme case to understand the intuition; e.g., if all purchases and recommendations were to come from a single customer.
What does total recommended products means what if treatment one is able to recommend tax more products and the purchase rate out of these recommended product is X2 system one the control group is able to recommend only 1/10 of the product and the purchase rate is half of the system too so in this case the purchase rate for system one is higher but more users have purchased in the total revenue is higher in system B
Z-tests are statistical calculations that are generally used to compare population mean to a sample mean. The z-score tells how far, in terms standard deviations, a data point is from the mean or average of a data set. So a z-test compares a sample to a defined population. Like z-tests, t-tests are calculations used to test a hypothesis, but they are used mostly when we need to determine if there is a statistically significant difference between two independent sample groups. In other words, a t-test asks whether a difference between the means of two groups is unlikely to have occurred because of random chance.
I think Conversion rate might be a better primary metric here , across the variants.
high level: click thru rate uplift, goal converstion uplift (i.e. video view in ott, purchase in ecomm), etc etc
lower level: mean average precision, AP at k, AR at k
@@AlphaMaleMy what AP, AR?
I wish you had mentioned the feedback would be on another website at the beginning of the interview. Also, is this person who is being interviewed a real candidate or someone who is already working as a DS at Amazon (or somewhere else?)
He A/B tested it and found out that more peole will watch the video, and more people will move to paid subscription.
Major dick move, but this part isn't measured and not important.
I think it would be better if we take the metric as:
# recommended products purchased / total # of recommended products
Even I thought the same. It would make more sense
Although it would seem more straightforward at first blush, it wouldn't be a particularly favorable metric in this case. See, if a certain user makes several searches and/or purchases (which is inevitable, in practice), the underlying observations that contribute to the metric are correlated with one another, leadning to a violation of the t-test's assumption of observations' independence. Think of an extreme case to understand the intuition; e.g., if all purchases and recommendations were to come from a single customer.
What does total recommended products means what if treatment one is able to recommend tax more products and the purchase rate out of these recommended product is X2 system one the control group is able to recommend only 1/10 of the product and the purchase rate is half of the system too so in this case the purchase rate for system one is higher but more users have purchased in the total revenue is higher in system B
*Where can I find more resources on AB Testing interview questions?*
1 ctr. 2 # of keywords searched 3 whether use bundle sales
What a massive waste of time. How is this video supposed to add absolutely any value to anyone? Stop clickbaiting.
where is the feedback? I can not find.
What about a high NDCG score ?
I really need mentor like you for data analytics as I m interested in analytics 🙂
Can you please upload a video on how to do a t test? it's urgent please.
Is the interviewing person Daliana Liu? :)
why t test not z test
same question..
Z-tests are statistical calculations that are generally used to compare population mean to a sample mean. The z-score tells how far, in terms standard deviations, a data point is from the mean or average of a data set. So a z-test compares a sample to a defined population.
Like z-tests, t-tests are calculations used to test a hypothesis, but they are used mostly when we need to determine if there is a statistically significant difference between two independent sample groups. In other words, a t-test asks whether a difference between the means of two groups is unlikely to have occurred because of random chance.
Isn't it a binomial test (2 sample version) since her metric is in proportions. T test, afaik, is for means (which distribute normally using clt).
Yes, you are correct. z-test is the correct test stat to use
Good bad example case :/
Oh my god, “uh uh uh uh uh uh uh” please stop saying “uh”
I am sorry, I have found a lot of good content in this channel but this video has to be the worst one.
Dont bother watching, no feedback at the end and you will waste 16 minutes
this lady needs to improve her English speaking