Bulk reprocessing and republishing of videos (DM enabled) | Community
Skip to main content
Vishal_Anand
Vishal_Anand
Level 4
February 8, 2026
Question

Bulk reprocessing and republishing of videos (DM enabled)

  • February 8, 2026
  • 2 replies
  • 14 views

Hi Team — need advice on bulk video reprocessing and republishing (5k+ assets)

Background: I need to reprocess videos across multiple folders. I have two options:

  • Run CURL commands in batches (preferred — keeps processing external to platform; Adobe recommended this).
  • Run a Groovy script inside AEM.

Questions / requests:

  1. Republishing strategy — recommended batch size, pacing, and concurrency controls? Reprocessing is async, so what safe concurrency limits or delay patterns have you used to avoid overloading Dynamic Media/encoders?
  2. Is there an OOTB AEM workflow or pattern to trigger bulk video republishing across folders (e.g., adapt DAM Update Asset, Workflow Launcher, or Dynamic Media jobs)? If yes, can it be run cross-folder and at scale?
  3. Error handling & idempotency — advice on retry/backoff strategy, transient vs permanent error handling, and marking completed assets to avoid double-processing?
  4. Any known platform limits, performance caveats or recommended throttling values (per minute/hour) for Dynamic Media/encoding ingestion when processing thousands of videos?

Thanks

    2 replies

    Vishal_Anand
    Vishal_AnandAuthor
    Level 4
    February 9, 2026

    Adding curl command for reference:
     

    FOR /F "usebackq delims=" %A IN ("batch1.txt") DO curl --ssl-no-revoke -u "admin:xxxx" -X POST "https://author-xxxxxx.adobeaemcloud.com/bin/asynccommand" -d "operation=PROCESS" -d "processType=full" -d "description=Selective reprocess – batch 1/x" -d "asset=%A"

     

    giuseppebaglio
    giuseppebaglio
    Level 10
    February 9, 2026

    hi ​@Vishal_Anand 

    AFAIK no hard per-minute/hour quotas is published published , the recommended batch sizes are 100 videos max per job, with 2-5 minute delays between batches to prevent Dynamic Media encoder overloads. KCS guidance stresses processing no more than 2,000 videos total without pauses, as larger bursts cause high load and failures.

    No single OOTB workflow spans cross-folder bulk video republishing, so you can add publish action as next step to be executed during pause between executions.

    For errors, you could mark completed assets via custom metadata which you can later query via AEM Assets HTTP API to filter out successes and avoid double-processing.
    For retries, I would implement exponential backoff distinguishing transient errors (e.g., busy: wait/retry) from permanent. You can use API to list failed assets before each batch.

     

      A
      Abie
      Level 4
      February 9, 2026

      Hi ​@Vishal_Anand ,

      ​@giuseppebaglio has covered quite a lot of points, much appreciated!
      A couple of practical aspects that we considered during such a situation-

      1. A batch size could depend on the size of the videos/assets. A large batch of small sized videos as compared to large batch of medium or large size videos. We created these batches in pre-migration/processing phase and executed batches accordingly.
      2. Execute these re-processings during non-peak hours.

      Lately, we re-processed the entire DAM (about 50k assets) with about 5-7 videos during the weekend and it worked like a charm. 

       

      Thanks!

        Terms & ConditionsAccessibility statement

        Loading footer...