How simulations outperform traditional statistics because they are easier to understand, more flexible and economically meaningful
Controlled experiments, such as A/B testing, are widely used by companies.
However, many people are put off by A/B testing because of the presence of intimidating statistical jargon that includes terms like “confidence,” “power,” “p-value,” “t-test,” “effect size,” etc.
In this article I will show you that you don't need a Master's degree in Statistics to understand A/B testing, but quite the opposite. In fact, simulations can replace most of those statistical artifacts that were necessary 100 years ago.
Not only that: I will also show you that the viability of an experiment can be measured using something that, unlike “trust” and “power,” is understandable to anyone in the company: dollars.
Your website has a checkout page. The ML team has created a new recommendation model. They claim that by incorporating their recommendations into the checkout page, we can increase revenue by a staggering 5%.
