Bulk upload/import assets in AEM 6.5

[girishb83316758]

Hi All

I have a requirement to upload 15k files in /content/dam folder. In this case what is the best solution to achieve this.?

I have read about bulk import option, but the option is not visible in AEM 6.5. (How to get this feature)

Appreciate any ideas or approaches here.

Thank you,

@Jörg_Hoh 

@lukasz-m 

[lukasz-m · Accepted solution]

Hi @girishb83316758,

Asset bulk is not available in on-premise version like AEM 6.5, so it can not be enabled or installed. However there are other tools that will help you.

I think it is worth to explore CSV Asset Importer from ACS AEM Tools:

• https://adobe-consulting-services.github.io/acs-aem-tools/features/csv-asset-importer/index.html

and/or Data Importer from ACS AEM Commons:

• https://adobe-consulting-services.github.io/acs-aem-commons/features/mcp-tools/data-importer/index.h...

I would strongly recommend to read below guide:

• https://experienceleague.adobe.com/docs/experience-manager-65/assets/administer/assets-migration-gui...

It describes holistic approach related to large amount of asset upload to AEM using above tools. In my opinion looking into above article should be your first step, because asset upload is only one aspect you have to do, the other are rendition generation (and how to to it smart etc), metadata handling etc.

[girishb83316758]

Hi,

We are using Asset API to upload assets in AEM.

We are uploading around 15K files. While uploading, DAM Update asset is executing. Because of huge volume. AEM ran out of memory. And we DAM Update assets workflow executed at the time of uploading itself.

Any configuration/ways we can increase the OOTB DAM update workflow to handle 15K+ assets processing without AEM running out of memory issues.?

@lukasz-m 

@Jörg_Hoh 

Thanks,

[lukasz-m]

Hi @girishb83316758,

According to best practices, you should disable DAM Update Asset workflow until assets upload is completed. After that, you should use ACS Commons Bulk Workflow Manager to run DAM Update Asset workflow in controlled manner.

Once again I would suggest, that you will first go through below guide from Adobe:

• https://experienceleague.adobe.com/docs/experience-manager-65/assets/administer/assets-migration-gui...

It covers all the aspects you need, and provides knowledge that would allow you to avoid issues like you have right now - out of memory.

I think you should do a step back and prepare proper plan for asset upload, base on above Adobe documentation.

[girishb83316758]

I have gone through the references, Thank you.

Can we upload assets in a batch of 500 or 1000 with DAM update asset workflow enabled with some time intervals of 30 mins before each run.? Since this will  be a one time activity. I would like to try this approach, Your thoughts on this.?

@lukasz-m 

Thanks,

[Jörg_Hoh]

If you do all these steps in a more controlled manner (for example by checking the AEM state and potentially waiting before uploading the next asset), it can work as well, but probably the overall throughput will decrease.

Depends on your general priorities regarding the time this process is allowed to take, the time investment to actually make it work and others.