This is entirely dependent on the project. Some projects may take weeks or months to roll out and others may take just a few days. For context, Exponent regularly tests new thumbnails and titles for our content on UA-cam and our website to see how it impacts learning journeys. We usually let these tests run for about 2 weeks before we have enough statistically significant data. Other tests, like changing a navigation element, may take 12 weeks to get enough data on because of the number of users who use the feature may be far less. You run experiments until you achieve a statistically significant result with high-predictability of outcome. Sometimes that's done with a gut-check. Sometimes you use statistical modeling tools to tell you if your experiment needs more time.
Noted! If you know any product designers who'd like to sit down with us, let us know! More product design videos already scheduled this month. Stay tuned!
Hopefully you see this comment. Could this framework be used in other instances, or would you recommend a different video for that? Not just an interview.
Don't leave your product management career to chance. Sign up for Exponent's PM interview course today: bit.ly/3uteTzQ
The best video I found for explaining this concept.
Love it... Just what I was looking for..
Perfect!
Amazing, Clearly explained everything.
Dang! Your video is so clean
Thank you. Great content!
Loved the detailed information.
Well explained within 5 minutes!!!
I gave the 1000th like 🎉
How do you decide that ( time) till when you want to run the experiment?
This is entirely dependent on the project. Some projects may take weeks or months to roll out and others may take just a few days.
For context, Exponent regularly tests new thumbnails and titles for our content on UA-cam and our website to see how it impacts learning journeys. We usually let these tests run for about 2 weeks before we have enough statistically significant data.
Other tests, like changing a navigation element, may take 12 weeks to get enough data on because of the number of users who use the feature may be far less.
You run experiments until you achieve a statistically significant result with high-predictability of outcome. Sometimes that's done with a gut-check. Sometimes you use statistical modeling tools to tell you if your experiment needs more time.
Great content! Would love to see more of these targeted to product designers and the rest of the design process.
Noted! If you know any product designers who'd like to sit down with us, let us know!
More product design videos already scheduled this month. Stay tuned!
Hopefully you see this comment. Could this framework be used in other instances, or would you recommend a different video for that? Not just an interview.
nice
What about MRE, alfa, beta? Too superficial to be useful.