Upload a lot of Assets into DAM lead to AEM crash

[Jacleng]

Environment: AEM 6.0 + sp2 + hotfix-6167 + hotfix-5918 + hotfix-6109

I used AssetManager API to upload about 50thousand asset into AEM instance, it will crash soon(Total 400GB, upload about 30GB, then crash).

Does anyone have suggestion about this?

Thanks in advance!

Kelvin

[Pawan-Gupta · Accepted solution]

Hello,

It always adviced to upload DAM assets in batch because most of the time when you have upload assets and if some assets which you are uploading are big so usually it takes time to transfer over the net and sometime it leads to expire the session or break the communication (as per my experience)

Second, avoid bundling bigger files as part of upload instead upload them separately.

Thanks,

Pawan

[leeasling]

It's probably crashing because of the workflows that are running to create renditions and extract meta data.

I'd recommend taking a look at offloading the jobs:

http://docs.adobe.com/docs/en/aem/6-0/deploy/configuring/offloading.html#Offloading the Processing of DAM Assets
http://dev.day.com/content/ddc/en/gems/introduction-of-job-handling-and-offloading-in-aem-5-6-1-/_jc...

[smacdonald2008]

Open a ticket for this and determine if you have reached the limit of the AEM DAM or if a hotfix address this. 

[Jacleng]

Late to reply.

Thank you! 🙂

[Jacleng]

Hi,

The reference is pretty good, thanks a lot.

Offloading distributes processing tasks amoung instances, now I only have one instance, and need to upload all the assets into it. But I think that I can control the task start or not to make the server stable, but now have no time to validate it, I'll try it later.

best regards,

Kelvin

[Jacleng]

Hi Pawan,

I tried to upload the asset in batch, controlled the upload speed. it worked fine, 

thanks for your help,

best regards,

Kelvin