Asset file split/chunk send over REST API - DAM | Community
Skip to main content
Nitin_laad
Community Advisor
Nitin_laadCommunity Advisor
Community Advisor
January 17, 2020
Solved

Asset file split/chunk send over REST API - DAM

  • January 17, 2020
  • 2 replies
  • 5212 views

Hello Everybody, 

 

We have a requirement to push assets from AEM DAM to external DAM (non AEM-DAM) using rest API.
It is easy to push small size assets (up to 20 MB), but issue is with larger size assets due to Rest API data transfer limit.
One option is to split the large files in chunks and send each chunk over Rest api call.

but in case of bulk asset push what should be right approach to handle these assets chunks 

 

1. InputStream
2. File system (temp file)
3. or any other recommandation (please)

In case of inputStream, lets say for one assets (size = 50 mb), we have created 3 chunks, chunk push on Rest API should be sequential or parallel.
Sequential processing of chunks will take more time, whereas parallel processing will take less time, but  multi threading is having its own overhead. 

 


can you please suggest recommended way to handle this kind of use case.

 

Regards, 

Nj

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by joerghoh

It totally depends on the remote system, what is the best way to make it work. Technically you can write code, which reads the data from the binary asset object, and sends it chunks of 20MB to the remote system. You just need to know if (and how) the remote system can handle that a new asset is sent in chunks.

 

Another way would be a transfer file share, where your custom code in AEM will export the files into and from where the remote system is reading the data then.

 

It's hard to recommend a specific approach without knowing the details of the affected products and all the circumstances and requirements around it.

2 replies

L
Adobe Employee
leoberliantAdobe Employee
Adobe Employee
January 17, 2020

You need to configure the chunkUploadMinFileSize property of the fileupload node in the CRX repository.

Please review:

https://docs.adobe.com/content/help/en/experience-manager-64/release-notes/sp-release-notes.html

and

https://helpx.adobe.com/ca/experience-manager/6-4/assets/using/managing-video-assets.html#Configurationtouploadvideoassetsthatarelargerthan2GB

Nitin_laad
Community Advisor
Nitin_laadCommunity AdvisorAuthor
Community Advisor
January 18, 2020

Thank you Berliant for your time and reply.

 

I am not facing any asset size issue in uploading the asset in AEM DAM, but while pushing it to an external system using Rest API. 

Any reference material on that type of implementation would be a great help. 

 

Regards, 

Nj

joerghoh
Adobe Employee
joerghohAdobe EmployeeAccepted solution
Adobe Employee
January 19, 2020

It totally depends on the remote system, what is the best way to make it work. Technically you can write code, which reads the data from the binary asset object, and sends it chunks of 20MB to the remote system. You just need to know if (and how) the remote system can handle that a new asset is sent in chunks.

 

Another way would be a transfer file share, where your custom code in AEM will export the files into and from where the remote system is reading the data then.

 

It's hard to recommend a specific approach without knowing the details of the affected products and all the circumstances and requirements around it.

Nitin_laad
Community Advisor
Nitin_laadCommunity AdvisorAuthor
Community Advisor
January 23, 2020
Thank you Jorg for your time and reply. Remote system is able to merge asset chunks into one. I had posted this question in forum to get some reference material, if someone already implemented this type of use case.
Terms & ConditionsAccessibility statement

Loading footer...