AEM Author is going down - Asset migration

[Satya_adobe]

Hi Guys,

We are in between a asset migration where we are using webdav tool(bitkinnex) to migrate assets to AEM author server(AEM 5.6.1). 
We have migrated around 11 Gigs of assets until now and as obvious there are lot of workflows are being trigged in background.

But the issue we are facing over here is although we have stopped the migration before 1 day but still the back-end workflows are running and some times the AEM is going down. 

Below is my AEM start up configuration.

CQ_JVM_OPTS='-server -Xmx2048M -XX:PermSize=256M -XX:MaxPermSize=1024M -Djava.awt.headless=true  -XX:+HeapDumpOnOutOfMemoryError 

Server details :- quad core processor, 10GB RAM

So need yours valuable input on this.

[askdctm]

Hi Satya,

- Are you seeing any Out of memory errors?

- Are the workflows going stale?

- Any suspicious message getting logged ,what do thread dumps point to?

[Opkar_Gill]

Hi Satya,

I am presuming this is a test upload? Can you reassess your approach?

The memory you have allocated seems low. Review the documentation on DAM performance: https://docs.adobe.com/docs/en/cq/5-6-1/dam/dam_performance_sizing.html

Adobe Managed Services documented some tests they ran for asset ingestion: http://cq-ops.tumblr.com/post/56244733735/ams-minions-dam-asset-ingestion-performance

You want to either throttle the upload of assets to allow the DAM asset upload workflows to complete, or disable these workflows, upload the assets and then write a script to cycle through the assets to initiate the workflows: https://edivad.wordpress.com/2013/01/07/installing-big-packages-in-cq5crx/

[leeasling]

Offloading is your best bet.

On another note, with doing such a large migration i would increase the amount of RAM you're allowing the instance to use.  I would set 4096 and 512 respectively in your startup parameters.

[francisco_ribei]

One possible solution would be offloading:

http://docs.adobe.com/docs/en/aem/6-0/deploy/configuring/offloading.html#Offloading the Processing of DAM Assets

[Opkar_Gill · Accepted solution]

http://docs.adobe.com/docs/en/aem/6-0/deploy/configuring/offloading.html

Offloading is an alternative, but depends on your requirements. Is this amount of assets being uploaded a one off for a migration or a regular occurrence?