Expand my Community achievements bar.

Join us January 15th for an AMA with Champion Achaia Walton, who will be talking about her article on Event-Based Reporting and Measuring Content Groups!
RSVP Here
SOLVED

Data Warehouse Reporting connects to server FTP but does not upload files

Avatar

Level 1
Level 1
6/20/18 7:46:30 AM

Greetings,

We have about 100 reports scheduled to run every night for about a year which were working fine until a couple days ago.

Today and yesteday only 25% of all reports were delivered to our server.

The rest are failing with the following message

Information
Reason:This report failed to successfully transfer.
Note:The transfer process may have left behind incomplete report files on the destination server.

We are seeing the following information in our logs (actual username was replaced with *FTP_USERNAME*):

SESSION|06/20/2018 06:12:09.977|[FTP:lookup:21][1638214] Accepting connection from: 192.243.242.55:33434

SESSION|06/20/2018 06:12:10.070|[FTP:1638214::192.243.242.55] READ : *USER *FTP_USERNAME**

SESSION|06/20/2018 06:12:10.070|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *331 Username OK.  Need password.*

SESSION|06/20/2018 06:12:10.148|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *PASS*

SESSION|06/20/2018 06:12:10.180|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *230 Password OK.  Connected. logged in*

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *SYST *

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *215 UNIX Type: L8*

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *CWD data*

SESSION|06/20/2018 06:13:54.760|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *250 "/data/" CWD command successful.*

SESSION|06/20/2018 06:13:54.776|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] READ : *QUIT *

SESSION|06/20/2018 06:13:54.776|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] WROTE: *221 Goodbye.*

SESSION|06/20/2018 06:13:54.776|[FTP:1638214:*FTP_USERNAME*:192.243.242.55] *Disconnected.*

SESSION|06/20/2018 06:12:10.148|[FTP:lookup:21][1638215] Accepting connection from: 192.243.242.54:45566

SESSION|06/20/2018 06:12:10.226|[FTP:1638215::192.243.242.54] READ : *USER *FTP_USERNAME**

SESSION|06/20/2018 06:12:10.242|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *331 Username OK.  Need password.*

SESSION|06/20/2018 06:12:10.320|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *PASS*

SESSION|06/20/2018 06:12:10.336|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *230 Password OK.  Connected. logged in*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *SYST *

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *215 UNIX Type: L8*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *CWD data*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *250 "/data/" CWD command successful.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] READ : *QUIT *

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] WROTE: *221 Goodbye.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638215:*FTP_USERNAME*:192.243.242.54] *Disconnected.*

SESSION|06/20/2018 06:12:10.680|[FTP:lookup:21][1638216] Accepting connection from: 192.243.242.52:28409

SESSION|06/20/2018 06:12:10.773|[FTP:1638216::192.243.242.52] READ : *USER *FTP_USERNAME**

SESSION|06/20/2018 06:12:10.773|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *331 Username OK.  Need password.*

SESSION|06/20/2018 06:12:10.852|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *PASS*

SESSION|06/20/2018 06:12:10.883|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *230 Password OK.  Connected. logged in*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *SYST *

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *215 UNIX Type: L8*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *CWD data*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *250 "/data/" CWD command successful.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] READ : *QUIT *

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] WROTE: *221 Goodbye.*

SESSION|06/20/2018 06:13:54.760|[FTP:1638216:*FTP_USERNAME*:192.243.242.52] *Disconnected.*

This issue began happening very recently.

Server has plenty of storage available, more than enough bandwidth and compute to handle 100x the capacity of current reporting and had 100% uptime for FTP service during attempted data transfer.

Is there a place where I can get more meaningful information about the cause of this error or something that I can do on my side to ensure delivery of reports?

Views

5.0K

Replies

7
Sign in to like this content

Total Likes

Translate
Translate
1 Accepted Solution

Avatar

Correct answer by
Level 1
Level 1
6/21/18 12:11:31 PM

All files were delivered successfully today, indicating that there is no issue with server configuration.

All non-adobe data transfers worked fine while adobe reports were failing to deliver.

Files that were scheduled for delivery but did not arrive had plenty of data in them, so it was not the cause.

In order to make sure that this issue does not happen again, we decided to duplicate ALL scheduled reports and run them multiple times for the previous day, ensuing that even something funky happens with adobe servers at 3am, another attempt to deliver files will be made at 5am.

If someone has a better solution than one that we decided on, please post it here.

Thanks everyone!

View solution in original post

Views

3.4K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate
Jump to reply
7 Replies

Avatar

Community Advisor
Community Advisor
6/20/18 11:15:56 AM

Can you verify that none of the scheduled files were given the same name? Could cause the ftp to not write a file if multiple same named files are attempted.

Views

3.1K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 1
Level 1
6/20/18 11:19:33 AM

All scheduled files have different names with %R token to ensure uniqueness of files.

Views

3.2K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Community Advisor
Community Advisor
6/20/18 11:51:36 AM

What if you manually login delete all present files. The message references incomplete files maybe a read/write error on ftp server. Some network interruption may have corrupted the files and now rewriting them isn't possible, causing new uploads to fail.

Views

3.1K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 1
Level 1
6/20/18 12:13:58 PM

Message that references incomplete files is a placeholder that is added to every failed data transfer. If you look at the logs, you will see that client connected to the FTP server, logged in with correct credentials and simply disconnected instead of even attempting to transfer files.

We have a monitoring service(hosted externally) that was able to communicate with FTP server during attempted file transfer (therefore, not a network outage).

Views

3.2K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Level 4
Level 4
6/21/18 11:49:35 AM

Looks like something is up with FTP,

Few thing that I would suggest you to validate

1. try with any other ftp location/account

2. try moving some files in the ftp create a new folder and move in and out of it

3. check with request ID's in Data warehouse request manager to validate with the errors or you can check with the client care team to share the logs if it looks like an issue with Adobe schedule delivery.

4. test some immediate deliveries to the FTP

Views

3.7K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Community Advisor
Community Advisor
6/21/18 11:59:09 AM

I just thought of something else.... Is it possible that some of the scheduled reports had no data to report? Could things like pagename or variable/segment names have changed causing 0 results?

Views

3.1K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate

Avatar

Correct answer by
Level 1
Level 1
6/21/18 12:11:31 PM

All files were delivered successfully today, indicating that there is no issue with server configuration.

All non-adobe data transfers worked fine while adobe reports were failing to deliver.

Files that were scheduled for delivery but did not arrive had plenty of data in them, so it was not the cause.

In order to make sure that this issue does not happen again, we decided to duplicate ALL scheduled reports and run them multiple times for the previous day, ensuing that even something funky happens with adobe servers at 3am, another attempt to deliver files will be made at 5am.

If someone has a better solution than one that we decided on, please post it here.

Thanks everyone!

Views

3.4K

Replies

0
Sign in to like this content

Total Likes

Translate
Translate
Jump to reply
Related Conversations
How to configure datastream id dynamicaly Solved

Views

144

Likes

0

Replies

5
Report suite name set in Adobe websdk implementation Solved

Views

87

Likes

0

Replies

4
Not able to create a new AA account for this email: ian.scott-mance@mt.com

Views

15

Likes

0

Replies

1
How to analytic the Feed Ads attribution?

Views

38

Likes

0

Replies

1
What's the exact syntax for populating data layer elements in Adobe Launch using ACDL?

Views

117

Like

1

Replies

4