How to judge a Subject Line Test Result: Opens vs Click to Opens? | Community
Skip to main content
A
Anonymous
January 20, 2015
Solved

How to judge a Subject Line Test Result: Opens vs Click to Opens?

  • January 20, 2015
  • 5 replies
  • 2495 views
We just ran a subject line test on 5 subject lines. The results have a clear leader for the most opens on Subject Variant "C".  However, Subject Variant "E" had a lower open rate, but a drastically higher Click to Opens rate.

Any advice on which is the better metric to based a final decision on which variant we should send?

Results here:
This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Kristen_Malkov1
Also keep in mind the click to open ration is simply the number of people that clicked on that email that opened the email first. Since E was in the lowest for opens, it would make sense that the ratio of click to open would be higher. 

5 replies

Kristen_Malkov1
Kristen_Malkov1
Level 7
January 21, 2015
Did you provide a different CTA for varient E? If the CTA was the same in all emails, I'd say that you should stick with the highest number of opens. I'm not personally a fan of the Click to Open ratio, but a lot of it is based on personal preference. Also, it might be better to do an additional test with a higher number of leads.
Kristen_Malkov1
Kristen_Malkov1Accepted solution
Level 7
January 21, 2015
Also keep in mind the click to open ration is simply the number of people that clicked on that email that opened the email first. Since E was in the lowest for opens, it would make sense that the ratio of click to open would be higher. 
A
Anonymous
January 21, 2015
Is it even possible to do a subject line test and provide different CTA options? I thought that a Subject Line test only allows for subject lines to change, but the exact same email.  

Good point on the ratio. Open rate seems the right way to go. 
Kristen_Malkov1
Kristen_Malkov1
Level 7
January 21, 2015
Yeah you could also change up the CTA with the subject line. I tend to test both conurrently, and have them be somewhat related. A good site to check out is "which test won" if you're not familiar. They do regular testing of landing pages and email CTAs, might give you some good ideas for your next test.
Dory_Viscoglio
Dory_Viscoglio
Level 10
January 21, 2015
The only thing I'd add here, is that sometimes a subject line could be much more indicative of what someone will receive in the email which could lead to the higher click rate... I could promise people a million dollars for opening which would result in a high number of opens, but obviously a low number of clicks. On an email where you got a substantially higher number of clicks, I'd ask what was so different about the subject line before declaring a winner based on opens alone.
Terms & ConditionsAccessibility statement

Loading footer...