Profile:
Statistical results can lead to misguided conclusions if a sampling group is not representative of the greater population under consideration.
Here we examine why understanding the mechanics that lie beneath the phrase ‘statistically significant’ is essential when conducting tests or defending a hypothesis.
Understanding hypothesis testing mechanics and how it translates to A/B or multivariate testing will help with the study design and reading results.
Here, we’ll demonstrate one of the more common misuses of A/B testing and introduce a better option when testing with multiple factors.