SMTP Envelope's From field vs SMTP protocol/handshake's MAIL From (was: How to set Return-Path when using ACS Commons Email API?)

[LinearGradient]

Hi,

We expect the Return-Path to be set to support-websites@foo.com.

We have tried two different configurations and neither one is working. First we had:

DefaultMailService.xml
<?xml version="1.0" encoding="UTF-8"?>

<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
          jcr:primaryType="sling:OsgiConfig"
          debug.email="{Boolean}false"
          from.address="DO-NOT-REPLY@foo.com"
          smtp.host="foo.com"
          smtp.password=""
          smtp.port="25"
          smtp.ssl="{Boolean}false"
          smtp.user=""/>

And email.template:
Return-Path: support-websites@foo.com

Subject: ${subject}

A form has been submitted on your website ${page} as shown below:

------------------------

${body}
-----------------------

And then we tested:
From: DO-NOT-REPLY@foo.com

Subject: ${subject}

A form has been submitted on your website ${page} as shown below:

------------------------

${body}
-----------------------

and
<?xml version="1.0" encoding="UTF-8"?>

<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
          jcr:primaryType="sling:OsgiConfig"
          debug.email="{Boolean}false"
          from.address="support-websites@foo.com"
          smtp.host="foo.com.au"
          smtp.password=""
          smtp.port="25"
          smtp.ssl="{Boolean}false"
          smtp.user=""/>

But the Return-Path received by recipients stays  DO-NOT-REPLY@foo.com.

Any ideas how the ACS Commons Email API should be confugured so that it sets Return-Path as support-websites@foo.com?

Thanks in advance.

[justin_at_adobe]

Hi,

The first part is definitely correct, although you shouldn't have a blank space between headers. See http://docs.adobe.com/docs/en/cq/current/javadoc/com/day/cq/commons/mail/MailTemplate.html.

How exactly are you testing this? If you are looking at the headers in the received email, that may be error prone as there are a number of layers between AEM and your mail client. I would suggest enabling debug logging and look at the traffic going from AEM to your SMTP server.

Regards,

Justin

[LinearGradient]

Hi Justin,

We still have not found a way to fix this problem. However I have more details now.

What we are looking for is the following SMTP conversation/payload. In short we want the FROM value for the handshake/protocol to be return-path@foo.com, but the From value in the envelope to be DO-NOT-REPLY@foo.com:

12:01:18 PM - SMTP server *:2525 started 12:01:21 PM - SMTP connection from localhost/127.0.0.1, new connection count: 1 12:01:21 PM - Server: 220 nexus ESMTP SubEthaSMTP null 12:01:21 PM - Client: ehlo [127.0.1.1] 12:01:21 PM - Server: 250-nexus 250-8BITMIME 250-AUTH LOGIN 250 Ok 12:01:21 PM - Client: mail FROM:<return-path@foo.com> 12:01:21 PM - Server: 250 Ok 12:01:21 PM - Client: rcpt TO:<whoever@yahoo.com> 12:01:21 PM - Server: 250 Ok 12:01:21 PM - Client: data 12:01:21 PM - Server: 354 End data with <CR><LF>.<CR><LF> 12:01:21 PM - Server: 250 Ok 12:01:21 PM - Client: quit 12:01:21 PM - Server: 221 Bye **************************** Content-Type: multipart/mixed; boundary="===============8791918472533545512==" MIME-Version: 1.0 Subject: Test Of Return Path From: DO-NOT-REPLY@foo.com To: whoever@yahoo.com --===============8791918472533545512== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit This is the body --===============8791918472533545512==--

Our configuration is:

<?xml version="1.0" encoding="UTF-8"?> <jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0" jcr:primaryType="sling:OsgiConfig" debug.email="{Boolean}true" from.address="return-path@foo.com" smtp.host="localhost" smtp.password="" smtp.port="2525" smtp.ssl="{Boolean}false" smtp.user=""/>

And our email template is:

From: DO-NOT-REPLY@foo.com Subject: ${subject} A form has been submitted on your website ${page} as shown below: ------------------------ ${body} ----------------------

However this leads to the following SMTP conversation:

01:24:30 PM - SMTP server *:2525 started 01:24:38 PM - SMTP connection from localhost/127.0.0.1, new connection count: 1 01:24:38 PM - Server: 220 nexus ESMTP SubEthaSMTP null 01:24:38 PM - Client: EHLO nexus 01:24:38 PM - Server: 250-nexus 250-8BITMIME 250-AUTH LOGIN 250 Ok 01:24:38 PM - Client: MAIL FROM:<DO-NOT-REPLY@foo.com> 01:24:38 PM - Server: 250 Ok 01:24:38 PM - Client: RCPT TO:<whoever@yahoo.com> 01:24:38 PM - Server: 250 Ok 01:24:38 PM - Client: DATA 01:24:38 PM - Server: 354 End data with <CR><LF>.<CR><LF> 01:24:38 PM - Server: 250 Ok 01:24:38 PM - Client: QUIT 01:24:38 PM - Server: 221 Bye Date: Wed, 25 Jun 2014 13:24:38 +1000 (EST) From: "DO-NOT-REPLY@foo.com" <DO-NOT-REPLY@foo.com> To: whoever@yahoo.com Message-ID: <515712947.11.1403666678359.JavaMail.foo@nexus> Subject: Email subject 2 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_9_876209372.1403666678356" ------=_Part_9_876209372.1403666678356 Content-Type: multipart/alternative; boundary="----=_Part_10_1804675948.1403666678357" ------=_Part_10_1804675948.1403666678357 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit A form has been submitted on your website ... as shown below: ------------------------ date-1: 10/10/2020 date-2: 10/10/2020 ----------------------- ------=_Part_10_1804675948.1403666678357 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit <html><body><pre>A form has been submitted on your website ... as shown below: ------------------------ date-1: 10/10/2020 date-2: 10/10/2020 -----------------------</pre></body></html> ------=_Part_10_1804675948.1403666678357-- ------=_Part_9_876209372.1403666678356--

So the From in the template is overriding the from.address of DefaultMailService.xml, which is not what we expect.

Any ideas how can we fix this?

Thanks.

[justin_at_adobe · Accepted solution]

I don't see any reference to the Return-Path header in your template. Therefore it won't be set.

The behavior you are describing with the From address is expected -- the configuration of the DefaultMailService is the default From address. In your case, you are providing a specific From address, so the default is not used.

There is currently no way in AEM's Mail Service component to specify a different envelope from address than a message from address. If this is important to you, I'd suggest filing a feature request via DayCare.

[LinearGradient]

Hi Justin,

According to the SMTP specification, the return path should not be set by the sender but it is something that is set by the recipient. Please see:

• http://en.wikipedia.org/wiki/Bounce_message
• http://www.ietf.org/rfc/rfc2821.txt

In particular:

The Return-Path is visible in delivered mail as header field Return-Path inserted by the SMTP mail delivery agent (MDA) (which is usually combined with a mail transfer agent, or MTA). The MDA simply copies the reverse path in the SMTP MAIL FROMcommand into the Return-Path. The MDA also removes bogus Return-Path header fields inserted by other MTAs; this header field is generally guaranteed to reflect the last reverse path seen in the MAIL FROM command.

Here's a Python script that simulates the behavior I am expecting:

#!/usr/bin/python import smtplib import pprint from email.MIMEMultipart import MIMEMultipart from email.MIMEText import MIMEText msg = MIMEMultipart() me = "no-reply@foo.com" you = "sender@yahoo.com" returnpath="return-path@foo.com" # Used in the SMTP From command msg['Subject'] = "Test Of Return Path" msg['From'] = me msg['To'] = you body="This is the body" msg.attach(MIMEText(body, 'plain')) # Send the message via our own SMTP server, but don't include the # envelope header. s = smtplib.SMTP('smtp-server-host') s.sendmail(returnpath, you, msg.as_string()) s.quit()

I think I should raise a bug via Day Care if this is not supported by the Email API.

Thanks.