Very useful. Here the question. I have a control version + a variant. What should be the value inside "Number of Offers Including Control" 1 or 2 ? That's not clear to me.
found that some practical issues are quite hard 1. how to calculate required sample size for ratio or quantile metric ? 2. if we already know that there are some correlations between our sample,thus,underestimating the variance, how to estimate required sample size correctly either for AA test or AB test
1.For RPV metric, there is a separate tab/sheet in the calculator. In case your KPI is based on some other metric like Visits, you can use the usual one and make changes accordingly (Daily Sample Size will be Visits) 2. It is always best practice to analyze data in isolation. Eg - two tests running on same page, their results have to be analyzed independent of each other. In general, we always use segments to isolate data
To understand this, concepts like Confidence level, statistical power need to be understood. Will recommend to go through Adobe documentation on same - ‘How long should I run an A/B test’
Thanks a lot for this video!
Glad video was helpful
Very insightful.. thanks for sharing 🙂
Thanks Rishabh
Very useful. Here the question.
I have a control version + a variant. What should be the value inside "Number of Offers Including Control" 1 or 2 ?
That's not clear to me.
It will be 2 (as it mentions "including control")
@@digitalanalyticsonthego great! thank you
found that some practical issues are quite hard
1. how to calculate required sample size for ratio or quantile metric ?
2. if we already know that there are some correlations between our sample,thus,underestimating the variance, how to estimate required sample size correctly either for AA test or AB test
1.For RPV metric, there is a separate tab/sheet in the calculator. In case your KPI is based on some other metric like Visits, you can use the usual one and make changes accordingly (Daily Sample Size will be Visits)
2. It is always best practice to analyze data in isolation. Eg - two tests running on same page, their results have to be analyzed independent of each other. In general, we always use segments to isolate data
HI Have a query why is that for lower lift of 5% it requires more weeks to complete than a larger lift % of 17.5 %
To understand this, concepts like Confidence level, statistical power need to be understood. Will recommend to go through Adobe documentation on same - ‘How long should I run an A/B test’
Statistical power is not a Type 2 error. In fact, 1- statistical power is a Type 2 error.
Refer the below article for any doubts experienceleague.adobe.com/docs/target/using/activities/abtest/sample-size-determination.html?lang=en