Adobe Analytics

Dimitri101 · 6/20/18

Greetings,

We have about 100 reports scheduled to run every night for about a year which were working fine until a couple days ago.

Today and yesteday only 25% of all reports were delivered to our server.

The rest are failing with the following message

Information

Reason:	This report failed to successfully transfer.
Note:	The transfer process may have left behind incomplete report files on the destination server.

We are seeing the following information in our logs (actual username was replaced with *FTP_USERNAME*):

SESSION|06/20/2018 06:12:09.977|[FTP:lookup:21][1638214] Accepting connection from: 192.243.242.55:33434

SESSION|06/20/2018 06:12:10.070|[FTP:1638214::192.243.242.55] READ : *USER *FTP_USERNAME**

SESSION|06/20/2018 06:12:10.070|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *331 Username OK. Need password.*

SESSION|06/20/2018 06:12:10.148|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *PASS*

SESSION|06/20/2018 06:12:10.180|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *230 Password OK. Connected. logged in*

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *SYST *

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *215 UNIX Type: L8*

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *CWD data*

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *250 "/data/" CWD command successful.*

SESSION|06/20/2018 06:13:54.776|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *QUIT *

SESSION|06/20/2018 06:13:54.776|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *221 Goodbye.*

SESSION|06/20/2018 06:13:54.776|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] *Disconnected.*

SESSION|06/20/2018 06:12:10.148|[FTP:lookup:21][1638215] Accepting connection from: 192.243.242.54:45566

SESSION|06/20/2018 06:12:10.226|[FTP:1638215::192.243.242.54] READ : *USER *FTP_USERNAME**

SESSION|06/20/2018 06:12:10.242|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *331 Username OK. Need password.*

SESSION|06/20/2018 06:12:10.320|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *PASS*

SESSION|06/20/2018 06:12:10.336|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *230 Password OK. Connected. logged in*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *SYST *

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *215 UNIX Type: L8*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *CWD data*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *250 "/data/" CWD command successful.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *QUIT *

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *221 Goodbye.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] *Disconnected.*

SESSION|06/20/2018 06:12:10.680|[FTP:lookup:21][1638216] Accepting connection from: 192.243.242.52:28409

SESSION|06/20/2018 06:12:10.773|[FTP:1638216::192.243.242.52] READ : *USER *FTP_USERNAME**

SESSION|06/20/2018 06:12:10.773|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *331 Username OK. Need password.*

SESSION|06/20/2018 06:12:10.852|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *PASS*

SESSION|06/20/2018 06:12:10.883|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *230 Password OK. Connected. logged in*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *SYST *

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *215 UNIX Type: L8*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *CWD data*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *250 "/data/" CWD command successful.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *QUIT *

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *221 Goodbye.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] *Disconnected.*

This issue began happening very recently.

Server has plenty of storage available, more than enough bandwidth and compute to handle 100x the capacity of current reporting and had 100% uptime for FTP service during attempted data transfer.

Is there a place where I can get more meaningful information about the cause of this error or something that I can do on my side to ensure delivery of reports?

Dimitri101 · 6/21/18

All files were delivered successfully today, indicating that there is no issue with server configuration.

All non-adobe data transfers worked fine while adobe reports were failing to deliver.

Files that were scheduled for delivery but did not arrive had plenty of data in them, so it was not the cause.

In order to make sure that this issue does not happen again, we decided to duplicate ALL scheduled reports and run them multiple times for the previous day, ensuing that even something funky happens with adobe servers at 3am, another attempt to deliver files will be made at 5am.

If someone has a better solution than one that we decided on, please post it here.

Thanks everyone!

View solution in original post

Pablo_Childe · 6/20/18

Can you verify that none of the scheduled files were given the same name? Could cause the ftp to not write a file if multiple same named files are attempted.

Dimitri101 · 6/20/18

All scheduled files have different names with %R token to ensure uniqueness of files.

Pablo_Childe · 6/20/18

What if you manually login delete all present files. The message references incomplete files maybe a read/write error on ftp server. Some network interruption may have corrupted the files and now rewriting them isn't possible, causing new uploads to fail.

Dimitri101 · 6/20/18

Message that references incomplete files is a placeholder that is added to every failed data transfer. If you look at the logs, you will see that client connected to the FTP server, logged in with correct credentials and simply disconnected instead of even attempting to transfer files.

We have a monitoring service(hosted externally) that was able to communicate with FTP server during attempted file transfer (therefore, not a network outage).

suneeshkachroo · 6/21/18

Looks like something is up with FTP,

Few thing that I would suggest you to validate

1. try with any other ftp location/account

2. try moving some files in the ftp create a new folder and move in and out of it

3. check with request ID's in Data warehouse request manager to validate with the errors or you can check with the client care team to share the logs if it looks like an issue with Adobe schedule delivery.

4. test some immediate deliveries to the FTP

Pablo_Childe · 6/21/18