Curl command timeout while creating the bigger package | Community
Skip to main content
T
tnik
June 27, 2024
Solved

Curl command timeout while creating the bigger package

  • June 27, 2024
  • 2 replies
  • 2057 views

Hi ,

 

 I am using curl command to build the package (eg 250 MB) , but it times-out when the package is big and throws 503 error , but the package keeps building in the background and finally i can see a proper build package in the package manager ,

 

Can we increase the timeout of curl command and also is it possible to print building logs in terminal  as we see in the package manager 

 

I have tried to increase the curl timeout using -m flag , but its not working

 

Any workaround for the same? 

 

Thanks,

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    Best answer by aanchal-sikka

    @tnik 

     

    I have worked on Curl commands to copy content from one Cloud to another. But, I had assured that filters are used wisely to keep the package sizes small.

     

    I had used Curl for complete process:

    - Create & Build

    - Download 

    - Upload & Install

     

    So, I see 2 choices

    - Create short, but multiple packages

    - Check for package status. Before moving to next command.

    -- Please check if preview command helps. I haven't tried it

      https://experienceleague.adobe.com/en/docs/experience-manager-65/content/sites/administering/operations/curl#package-management 

    -- I guess download fails until build succeeds. Keep a short delay before you download. And if it fails, try download after some time.

     

    I had written the script more than an year back. So, unable to recollect all the details.

    2 replies

    pulkitvashisth
    Community Advisor
    pulkitvashisthCommunity Advisor
    Community Advisor
    June 27, 2024

    Hi @tnik 
    Can you try the ootb curl params for your issue :
    Like https://everything.curl.dev/usingcurl/timeouts.html

    You can use below params with your cUrl request and then check.
    <cUrl command to build> --max-time 600 --retry 5

    600 = 600 seconds
    Let me know if this works.

      T
      tnikAuthor
      June 27, 2024

      hi @pulkitvashisth , 

      i tried this flag it does not hold the connection for 600 seconds , it fails after every 60 seconds and go to next try till the retry count exhaust 

       

      Thanks

        T
        tnikAuthor
        June 27, 2024

        the retry just keep on increasing the build count set by the package manager API

         

          aanchal-sikka
          Community Advisor
          aanchal-sikkaCommunity Advisor
          Community Advisor
          June 27, 2024

          @tnik 

           

          1. Use a request to create and initiate your task
          2. And then poll the system until this job is completed, then return the result.

          This is an asynchronous pattern, and it’s much more scalable when it comes to the amount of processing you can do in there.

          Of course you cannot use a single curl command anymore, but now you need to write a program to execute this logic

           

          https://cqdump.joerghoh.de/2023/11/19/if-you-have-curl-every-problem-looks-like-a-request/

          Aanchal Sikka
            T
            tnikAuthor
            June 27, 2024

            Hi @aanchal-sikka ,

             

            have you tried this in the past , actually my requirement is to create the multiple package of site pages which would be roughly less 300 MB and move them from one cloud instances to another across the org  via automation , be it CI/CD scripts or programmatically at AEM level and then install them.

             

            but not sure if this kind of content migration thing between two cloud instances in different org using automated package approach  can be done via workflows or sling jobs

            aanchal-sikka
            Community Advisor
            aanchal-sikkaCommunity AdvisorAccepted solution
            Community Advisor
            June 28, 2024

            @tnik 

             

            I have worked on Curl commands to copy content from one Cloud to another. But, I had assured that filters are used wisely to keep the package sizes small.

             

            I had used Curl for complete process:

            - Create & Build

            - Download 

            - Upload & Install

             

            So, I see 2 choices

            - Create short, but multiple packages

            - Check for package status. Before moving to next command.

            -- Please check if preview command helps. I haven't tried it

              https://experienceleague.adobe.com/en/docs/experience-manager-65/content/sites/administering/operations/curl#package-management 

            -- I guess download fails until build succeeds. Keep a short delay before you download. And if it fails, try download after some time.

             

            I had written the script more than an year back. So, unable to recollect all the details.

            Aanchal Sikka
              Terms & ConditionsAccessibility statement

              Loading footer...