Expand my Community achievements bar.

Take 10 minutes to complete an Adobe Target review on Gartner Peer Insights, and as a thank you for your time, you’ll have the option to receive a \$25 gift card!
SOLVED

## Does Target account for multiple family errors (Stats) ?

Level 2

Basically what the title says- a statistics question.

For everyone else interested. Multiple family errors (alpha inflation) happens when testing more than one variant to 95% confidence (p = .05). While each test individually has a 5% chance of Type I error, as a family grouping the error rate increased with more variants. (Formula is: 1-(1-alpha)^n where alpha is .05 and n is the number of tests). So if we are testing 8 variations (+ 1 default), the formula would be: 1-(1-.05)^8 = .34 or a 34% chance of seeing a false positive among our grouping if we are testing 8 variants.

Does Target account for this automatically, or do we have to apply a correction post-hoc (or as a priori)?

1 Accepted Solution

Employee

Our sample size calculator offers the ability to account for family-wise error using Bonferroni correction.

We also discuss pitfalls and their resolutions in this article: Nine Common A/B Testing Pitfalls and How to Avoid Them

2 Replies