Adobe Target - Confidence - 99% - Clarification

Avatar

Avatar
Validate 10
Level 3
prathiv23
Level 3

Likes

23 likes

Total Posts

55 posts

Correct reply

1 solution
Top badges earned
Validate 10
Validate 1
Shape 1
Boost 5
Boost 3
View profile

Avatar
Validate 10
Level 3
prathiv23
Level 3

Likes

23 likes

Total Posts

55 posts

Correct reply

1 solution
Top badges earned
Validate 10
Validate 1
Shape 1
Boost 5
Boost 3
View profile
prathiv23
Level 3

04-09-2019

Hi Experts,

Newbie here in Target :-).

We have setup a A/B campaign.

The experience seems to work. Nevertheless, there is one curious number, the confident rate one. After a day we got over 99% of confidence while the traffic on this website is not high enough to generate this kind of confident rate after a day. (refer attached screenshot)

I've gone through Adobe Target documentation but I'm not able to follow the concepts.

Adobe Link - Average Lift, Lift Bounds, and Confidence Interval

Please help.

Thanks,

PV

Adobe Target - Confidence.PNG

Replies

Avatar

Avatar
Shape 25
MVP
Andrey_Osadchuk
MVP

Likes

539 likes

Total Posts

973 posts

Correct reply

266 solutions
Top badges earned
Shape 25
Bedrock
Coach
Boost 500
Affirm 250
View profile

Avatar
Shape 25
MVP
Andrey_Osadchuk
MVP

Likes

539 likes

Total Posts

973 posts

Correct reply

266 solutions
Top badges earned
Shape 25
Bedrock
Coach
Boost 500
Affirm 250
View profile
Andrey_Osadchuk
MVP

04-09-2019

Regardless of what Confidence Level is after a day, consider to continue the test for at least a full week.

I wonder why the Orders metric indicates 0.12 and 0.05 (I would expect these numbers percent-formatted)? Does 1504 stand for the total number of orders?

Can you share the full screenshot?

Avatar

Avatar
Validate 10
Level 3
prathiv23
Level 3

Likes

23 likes

Total Posts

55 posts

Correct reply

1 solution
Top badges earned
Validate 10
Validate 1
Shape 1
Boost 5
Boost 3
View profile

Avatar
Validate 10
Level 3
prathiv23
Level 3

Likes

23 likes

Total Posts

55 posts

Correct reply

1 solution
Top badges earned
Validate 10
Validate 1
Shape 1
Boost 5
Boost 3
View profile
prathiv23
Level 3

04-09-2019

Thank you

Avatar

Avatar
Shape 25
MVP
Andrey_Osadchuk
MVP

Likes

539 likes

Total Posts

973 posts

Correct reply

266 solutions
Top badges earned
Shape 25
Bedrock
Coach
Boost 500
Affirm 250
View profile

Avatar
Shape 25
MVP
Andrey_Osadchuk
MVP

Likes

539 likes

Total Posts

973 posts

Correct reply

266 solutions
Top badges earned
Shape 25
Bedrock
Coach
Boost 500
Affirm 250
View profile
Andrey_Osadchuk
MVP

05-09-2019

Thanks for sharing. Now it's clear that 0.13 and 0.05 are 13% and 5% conversion rates. Pay attention to the date range, it indicates a month not a day mentioned in the initial question.

Avatar

Avatar
Give Back 50
Employee
Shruthi_N
Employee

Likes

272 likes

Total Posts

236 posts

Correct reply

64 solutions
Top badges earned
Give Back 50
Give Back 5
Give Back 3
Give Back 25
Give Back 10
View profile

Avatar
Give Back 50
Employee
Shruthi_N
Employee

Likes

272 likes

Total Posts

236 posts

Correct reply

64 solutions
Top badges earned
Give Back 50
Give Back 5
Give Back 3
Give Back 25
Give Back 10
View profile
Shruthi_N
Employee

05-09-2019

prathiv23​ Also when running A/B tests and reading confidence values there are some things you should keep in mind. Always use sample size calculator to determine how long you should run the test before looking at the data. Here are some resources that should be helpful.

Nine common A/B testing pitfalls and how to avoid them

How long should you run an A/B test?