Expand my Community achievements bar.

Join us for our second AMA on experimentation and personalization strategies with Target, occurring on June 3rd!

Mark Solution

This conversation has been locked due to inactivity. Please create a new post.

SOLVED

What's a good confidence level you're measuring for A/B tests?

Avatar

Level 2

Hey target community,

I recently launched a few A/B tests on our web pages regarding pages' layout and copy. Changes are pretty noticeable if you're a repeat visitor to the page.

However, what I've seen from the data (past few weeks) is that confidence level fluctuates a lot as we're collecting more data. For one test that has been running on a low to medium volume page, we haven't reached statistical significance for any of the challenger experience to either win or lose to the control experience.

I want to ask the community what is the confidence threshold you 're measuring for your web experiment initiatives? Do you have a specific use case where you would take some calculated risks (say an 85% of confidence) but still implementing the experience on the page?

Any thoughts or inputs are much appreciated.

LI

1 Accepted Solution

Avatar

Correct answer by
Employee

Hi Li, Nine Common A/B Testing Pitfalls and How to Avoid Them  and Plan an A/B Test are good resources to consider while planning your tests.

View solution in original post

2 Replies

Avatar

Correct answer by
Employee

Hi Li, Nine Common A/B Testing Pitfalls and How to Avoid Them  and Plan an A/B Test are good resources to consider while planning your tests.

Avatar

Level 2

Thank you. These are great resources. I'll give them a read for sure.