Workflow improvement :Digital asset management

[askdctm]

Hi All,

DAM asset upload workflows are generally too resource intensive. The system performance[AEM 5.6.1] is affected.

We also face performance problems when many dam upload workflows[ex:DAM update asset workflows] are being run by the users.

Would like to improve the performance.

Went through couple of links as below:

1]http://cq-ops.tumblr.com/post/33794096940/what-happens-when-a-pdf-is-uploaded-into-cq-dam

2]http://docs.adobe.com/docs/en/cq/current/deploying/performance.html#par_title_23

3]http://docs.adobe.com/docs/en/cq/current/deploying/configuring_osgi.html#par_title_104

Some understanding as below:

When any workflows are initiated, internally the workflow engine uses Apache Sling job queues
There are two types of queues 
- Granite Workflow Queue
- Granite Workflow External Process Job Queue

We have to configure these services to limit the maximum number of concurrently running workflow processes. 

There is Apache Sling Job queue configuration available under bundles, which has a property Maximum Parallel Jobs.

Doubts as below:

a] Went through DAM Update asset workflow example under "CONFIGURE THE QUEUE FOR A SPECIFIC WORKFLOW MODEL" in [2].
   But I was not quite clear as to what is being done here.

b] Also, under http://localhost:4502/system/console/slingevent , as mentioned in the article in [2],
   I was not able to find the below mentioned[OOTB AEM 5.6.1]

   com/adobe/granite/workflow/external/job/etc/workflow/models/dam/update_asset/jcr_content/model
   com/adobe/granite/workflow/job/etc/workflow/models/dam/update_asset/jcr_content/model
   com/adobe/granite/workflow/job/etc/workflow/models/dam-xmp-writeback/jcr_content/model 

c] Also, can you please let me know the system requirements[RAM,processors etc], DAM assets size or other system
   parameters that is required for optimal performance.

d] If anyone has implemented some other solution, please share it.

Any thoughts on the above will be really helpful.

[Sam205505050]

Job Offloading is a good option when there are many assets   

[askdctm]

Hi Sam,

Thanks a lot for your reply.

Currently we have a active/passive setup, wherein we have actively running author instance and a passive instance
where backup is taken and multiple publishers.

So, is it something like :

a] Offloading is same as clustering? 
b] Also, how do I get the total size of DAM assets in my repository.http://docs.adobe.com/docs/en/cq/current/deploying/performance.html#par_title_23
   has some details. Can you please let me know if there is any other easy way to do this.
c] Generally, how complicated is setting up the offloading environment.
d] What prerequistes/analysis/study we should do before proceeding ahead.

[askdctm]

Hi Sam,

Also, is there any other way like making some changes to the existing DAM workflows to improve performance/any other steps.

Any thoughts here would be helpful.

[Sam205505050]

personally I have not got any chance to work on offloading, someone else might be able to help you

[askdctm]

Hi Sam,

Thank you for your reply.

@ All

Any thoughts on the above will be really helpful.

[Sham_HC · Accepted solution]

askdctm wrote...

@ All

Any thoughts on the above will be really helpful.

• Watch the recording at http://dev.day.com/content/ddc/en/gems/introduction-of-job-handling-and-offloading-in-aem-5-6-1-.htm... and will help for your use case
• Clustering and offloading are different, the recoding above will help you
• The recording should help for your preperation
• Setting up is easy but getting entire ops team awareness might take some time
• For dam size you don;t get accurate one from  http://docs.adobe.com/docs/en/cq/current/administering/reporting.html#Disk%20Usage & need custom develop one. 

[askdctm]

Hi Sham,

Thank you for your reply.

At the end of the video, there is a small demo given. Will try that out on my machine and post back my queries/doubts.