A/B-testing the 'Optimize for AI Inbox'-feature

Hi all,

I have recently gained access to the AJO feature ‘Optimize for AI Inbox’-feature for e-mails, which should make the text version of the e-mail optimized for an ai e-mail inbox, ultimately resulting in higher engagement.

I find the feature super easy to use, and I like the use case as a simple step towards making our e-mail deliveries more AI-ready.

A feature like this is very easy and interesting to perform a split test on, so I have already setup 2 separate split tests.

Test 1: 

Bulk/One-off 50/50 split with around 50.000 customers in each treatment (Ecom sales e-mail)

Result: No significance in either open rate or CTOR.

Test 2:

Always on survey/customer satisfaction e-mail. 50/50 split.

Result: Low volume as of now, so too early to conclude.

Disclaimer: I’m situated in Denmark (so are the customers from the test), and I’m aware of differences in european versions of email clients such as Gmail, where we don’t have full roll-out of the AI inbox feature yet. So I’m keen to hear from any non-european peers if you have been able to see differences.

Question

Have any of you tested the feature or do you have any learnings or thoughts about it?

Looking forward to hearing from you!