I recently ran an A/B test on two versions of the same page - an english version and a chinese version - the results were pushed to Adobe Analytics. The results are the opposite i.e. the chinese version shows that the experience B (experiment) was the winner, whereas in the english version of the test, experience A (control) was the winner. The winner is based on a conversion metric..Point to note: the chinese version of the site receives significantly more traffic and conversion.
- In such cases, what are some of the recommended decision rules to consider for determining if the test was successful or not?
- The answer may be that it depends, but what is the thought process or better approach to deciding if the test was successful or not? Thanks!
Are these answers useful? Help other community members by marking useful answers as accepted.