Expand my Community achievements bar.

Bulk Import from Amazon S3 Failing

Avatar

Level 2

We are migrating assets from an Amazon S3 bucket to AEM. The "uploads" folder in S3 contains around 10K assets.

 

We have tested importing assets in our lower environments, and most of those imports were successful.

 

I followed the same procedure, with the same configuration, in PROD. The connection test passed, and the Dry Run looked as expected. However, every time I have ran the bulk import in PROD, it has failed.

 

In PROD, I have tried multiple configurations, using buckets with less assets in S3, but everything has failed so far. Most imports will get to about 9K assets, then show a list of failed assets (the Job History says 1K failed assets, but the list only contains 100 failed assets).

 

Since then, I ran an old import in STG again and was successful. We have compared the configs between the STG and PRD buckets and also enabled logs. But we have been unable to determine where the issue is.

Topics

Topics help categorize Community content and increase your ability to discover relevant content.

1 Reply

Avatar

Community Advisor

Hi, 

What are you using to migrate the assets? The logs should let you know why the assets failed to migrate. What happens if you migrate small batches? Can migrating small batches an option?



Esteban Bustamante