gatewaytimeout while creating package programatically in AEMasCS | Community
Skip to main content
raja-thannasi
Level 2
January 21, 2024

gatewaytimeout while creating package programatically in AEMasCS

  • January 21, 2024
  • 5 replies
  • 2275 views

Dear Developers/Architects,

 I was trying to create packages programatically in AEM, it fetchs given page's  path and page's reffrences like xf paths and dampaths(more than 1500 dampaths), i combined content path and xf path in seprate package(package-0). when comes to dampath i splited the dampath in to 300 dampath/package and tried to create  packages(package-1,package-2 & viceversa..).  when i ran this code in my local aem instance, i was able to create spilt of packages(combined (xf & content paths) package and split of dampath packages) without any issues. when i deploy the code on AEMasCS, im getting gateway timeout issue. Kindly let us know your suggestion on this...

java code if the dampath more than 300 

 

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

5 replies

Raja_Reddy
Community Advisor
Community Advisor
January 21, 2024

Hi @raja-thannasi 

Please check the permission in user admin for that user.if not,apply the permission to the user which ever required.

raja-thannasi
Level 2
January 22, 2024

Hi @raja_reddy , thanks for you're response. the code doesn't required any user permission, as im creating package only.not using any system user. So i feel permission may not issue. 

Kamal_Kishor
Community Advisor
Community Advisor
January 22, 2024

@raja-thannasi : How much of content is there on your AEMaaCS instance? If the size is too much, package creation can timeout, please refer this: https://experienceleague.adobe.com/docs/experience-manager-cloud-service/content/implementing/developer-tools/package-manager.html?lang=en#package-size

If possible, try to have less content for your testing to rule out if this is due to content size. In your case, you can try to reduce this to 50 or 100 from current 300 and see if it works.

yashp0808
Level 3
January 22, 2024

@raja-thannasi  can you try to save and close the session object in your finally block. 

raja-thannasi
Level 2
January 22, 2024

Hi @yashp0808 ,thanks for your response, the gateway timeout issue happened on the line below higlighted in the screenshot attached, where createPackage method. so saving the session will not work in the finally block.

 

 

 

@kamal_kishor , the size of the package was not able to size on package manager wizard,as it was incomplete state.please find the screenshot.

 

 

raja-thannasi
Level 2
January 22, 2024

Also here , on my Local AEM instance, the same code were working as expected able to package /content path. issue was with on AEMasCS environment. 

kautuk_sahni
Community Manager
Community Manager
January 23, 2024

@raja-thannasi Did you find the suggestions from users helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.

Kautuk Sahni
Raja_Reddy
Community Advisor
Community Advisor
January 23, 2024

Hi @raja-thannasi 
A good approach all we should have is a system user with required permissions. Here is the below code which achieves our requirement.

 

import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.util.HashMap; import java.util.Map; import javax.jcr.RepositoryException; import javax.jcr.Session; import javax.servlet.Servlet; import org.apache.jackrabbit.vault.fs.api.PathFilterSet; import org.apache.jackrabbit.vault.fs.api.ProgressTrackerListener; import org.apache.jackrabbit.vault.fs.config.DefaultWorkspaceFilter; import org.apache.jackrabbit.vault.packaging.JcrPackage; import org.apache.jackrabbit.vault.packaging.JcrPackageDefinition; import org.apache.jackrabbit.vault.packaging.JcrPackageManager; import org.apache.jackrabbit.vault.packaging.PackageException; import org.apache.jackrabbit.vault.packaging.PackagingService; import org.apache.jackrabbit.vault.util.DefaultProgressListener; import org.apache.sling.api.SlingHttpServletRequest; import org.apache.sling.api.SlingHttpServletResponse; import org.apache.sling.api.resource.ResourceResolver; import org.apache.sling.api.resource.ResourceResolverFactory; import org.apache.sling.api.servlets.HttpConstants; import org.apache.sling.api.servlets.SlingAllMethodsServlet; import org.osgi.framework.Constants; import org.osgi.service.component.annotations.Component; import org.osgi.service.component.annotations.Reference; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @Component(service = Servlet.class, property = { Constants.SERVICE_DESCRIPTION + "= API to download packages based on path", "sling.servlet.methods=" + HttpConstants.METHOD_GET, "sling.servlet.paths=" + "/bin/api/createpackage" }) public class PackagecreateServlet extends SlingAllMethodsServlet { private static final long serialVersionUID = 1L; private transient Logger log = LoggerFactory.getLogger(PackagecreateServlet.class); @Reference private transient ResourceResolverFactory resolverFactory; private void closePackageStream(InputStream packageStream) { if (packageStream != null) { try { packageStream.close(); } catch (IOException e) { log.error("Error while closing InputStream", e); } } } private InputStream getPackageStream(JcrPackage pack) throws Exception { if (pack != null && pack.getNode() != null && pack.getNode().hasProperty("jcr:content/jcr:data")) { return pack.getNode().getProperty("jcr:content/jcr:data").getBinary().getStream(); } return null; } @Override protected void doGet(SlingHttpServletRequest request, SlingHttpServletResponse response) throws IOException { String packageName = request.getParameter("packageName"); String packagePath = request.getParameter("packagePath"); JcrPackageManager jpm = null; if (packageName == null || packageName.isEmpty() || packagePath == null || packagePath.isEmpty()) { response.getWriter().write("failed: URL parameters 'packageName' or 'packagePath' are missing"); return; } String[] paths = packagePath.replace("[", "").replace("]", "").split(","); ResourceResolver resourceResolver = null; Session session = null; JcrPackage pack = null; InputStream packageStream = null; try { Map<String, Object> param = new HashMap<>(); param.put(ResourceResolverFactory.SUBSERVICE, "subservice"); resourceResolver = resolverFactory.getServiceResourceResolver(param); session = resourceResolver.adaptTo(Session.class); jpm = PackagingService.getPackageManager(session); pack = jpm.create("testPackage", packageName, "1.0.0"); JcrPackageDefinition definition = pack.getDefinition(); DefaultWorkspaceFilter filter = new DefaultWorkspaceFilter(); for (String path : paths) { PathFilterSet pathFilterSet = new PathFilterSet(); pathFilterSet.setRoot(path.trim()); filter.add(pathFilterSet); } boolean autoSave = true; if (definition != null) definition.setFilter(filter, autoSave); ProgressTrackerListener listener = new DefaultProgressListener(); jpm.assemble(pack, listener); packageStream = getPackageStream(pack); if (packageStream != null) { response.setContentType("application/zip"); response.setHeader("Content-Disposition", "attachment; filename=test-dynamic-package-1.0.0.zip"); OutputStream out = response.getOutputStream(); byte[] buffer = new byte[4096]; int bytesRead; while ((bytesRead = packageStream.read(buffer)) != -1) { out.write(buffer, 0, bytesRead); } out.flush(); } else { response.getWriter().write("failed: Package content is not available"); } } catch (PackageException e) { log.error("Package Exception: " + e.getMessage(), e); response.getWriter().write("failed: PackageException - " + e.getMessage()); } catch (Exception e) { log.error("Error while processing request", e); response.getWriter().write("failed: " + e.getMessage()); } finally { closePackageStream(packageStream); if (pack != null) { try { jpm.remove(pack); } catch (RepositoryException e) { log.error("Package Exception: " + e.getMessage(), e); } } if (resourceResolver != null && resourceResolver.isLive()) { resourceResolver.close(); } if (session != null && session.isLive()) { session.logout(); } } } }

 

URL how it looks like : http://localhost:portnumber/bin/api/createpackage?packageName=testpackage&packagePath=[/content/abcproject/abcpage,/content/abcproject/defpage,/content/abcproject/ghipage] After hitting this URL in browser a package gets downloaded into your local. we can add 'n' number of filters in the array of URL and customize the logic as per our requirement version we used is AEM 6.5. If we comment this piece of code jpm.remove(pack); the package is visible in /crx/packmgr/index.jsp.

 

 

 

raja-thannasi
Level 2
January 23, 2024

Hi @raja_reddy ,

 as suggested i updated the code but still not working and showing in error log as node path does not exist though its valid path.

Raja_Reddy
Community Advisor
Community Advisor
January 23, 2024

Hi @raja-thannasi 
Have you created a system user and provided permissions?

raja-thannasi
Level 2
January 24, 2024

Hi @raja_reddy , Please find the java file.

import java.io.IOException; import java.io.PrintWriter; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.HashMap; import java.util.HashSet; import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.Objects; import java.util.Set; import java.util.concurrent.atomic.AtomicInteger; import java.util.stream.Collectors; import java.util.stream.Stream; import javax.jcr.RepositoryException; import javax.jcr.Session; import javax.servlet.Servlet; import org.apache.commons.lang3.StringUtils; import org.apache.jackrabbit.vault.fs.io.AccessControlHandling; import org.apache.jackrabbit.vault.packaging.JcrPackage; import org.apache.jackrabbit.vault.packaging.JcrPackageDefinition; import org.apache.jackrabbit.vault.packaging.JcrPackageManager; import org.apache.jackrabbit.vault.packaging.PackageException; import org.apache.jackrabbit.vault.packaging.Packaging; import org.apache.sling.api.SlingHttpServletRequest; import org.apache.sling.api.SlingHttpServletResponse; import org.apache.sling.api.resource.Resource; import org.apache.sling.api.resource.ResourceResolver; import org.apache.sling.api.resource.ResourceResolverFactory; import org.apache.sling.api.servlets.SlingAllMethodsServlet; import org.apache.sling.servlets.post.JSONResponse; import org.jetbrains.annotations.NotNull; import org.osgi.service.component.annotations.Component; import org.osgi.service.component.annotations.Reference; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.adobe.acs.commons.packaging.PackageHelper; import com.adobe.granite.rest.Constants; import com.day.cq.wcm.api.Page; import com.day.cq.wcm.api.PageManager; import com.google.gson.Gson; import com.google.gson.JsonObject; /** * @author Raja * <p> * Servlet that facilitates the creation of content packages. This * servlet allows clients to create content packages by providing * relevant details. It extends the SlingAllMethodsServlet, allowing it * to handle various HTTP methods, primarily POST. */ @Component(service = { Servlet.class }, property = { "sling.servlet.paths=" + "/bin/triggerPackageCreator", "sling.servlet.methods=POST" }) public class PackageCreatorServlet extends SlingAllMethodsServlet { /** * */ private static final long serialVersionUID = 1L; // Logger instance for this class private static final Logger LOGGER = LoggerFactory.getLogger(PackageCreatorServlet.class); // Constants for package creation messages, default values and resource paths // List to store paths of child pages for the package private final List<String> childPagePathList = new ArrayList<>(); // Reference to a helper for package creation @Reference private transient PackageHelper packageHelper; // Reference to the packaging service @Reference private transient Packaging packaging; @Reference private ResourceResolverFactory resourceResolverFactory; /** * Handles the POST request to create a content package. Processes the input * parameters, interacts with the packaging service, and returns a JSON * response. * * @param request The Sling HTTP request object. * @param response The Sling HTTP response object. */ @Override protected void doPost(SlingHttpServletRequest request, SlingHttpServletResponse response) { response.setContentType(JSONResponse.RESPONSE_CONTENT_TYPE); response.setCharacterEncoding(Constants.DEFAULT_CHARSET); JsonObject jsonResponse = new JsonObject(); String packageName = request.getParameter(PageConstants.PACKAGE_NAME); String packageDescription = request.getParameter(PageConstants.PACKAGE_DESCRIPTION); String pathsList = request.getParameter(PageConstants.PACKAGE_LIST); String getChildren = request.getParameter(PageConstants.CHILD_PAGE_EXIST); int damNodeCount = Integer.parseInt(request.getParameter(PageConstants.DAM_NODE_COUNT)); int getPageLevel = request.getParameter(PageConstants.CHILD_PAGE_EXIST).equals(PageConstants.CHILD_PAGE_NO) ? 0 : Integer.parseInt(request.getParameter(PageConstants.PAGE_LEVEL)); List<String> packageArray = new ArrayList<>(); if (StringUtils.isNotEmpty(pathsList)) { packageArray = collectReferences(request.getResourceResolver(), pathsList, packageName, packageDescription, getChildren, getPageLevel, damNodeCount); } String msg; if (!packageArray.isEmpty()) { msg = "Package creation is Successful and path is"; for (String resourcePackagePath : packageArray) { msg = msg.concat("<br><a href=\"" + resourcePackagePath + "\">" + resourcePackagePath + "</a>"); } } else { msg = "Error"; } jsonResponse.addProperty(PageConstants.MESSAGE, msg); try (PrintWriter out = response.getWriter()) { out.print(new Gson().toJson(jsonResponse)); } catch (IOException e) { LOGGER.error("Unable to process package creation ...", e); } } /** * Collects references from a given set of paths and creates a package from * them. * <p> * This method identifies content references within the provided page paths and * aggregates them into different sets (e.g., experience fragments, digital * assets). Depending on the input parameters, it might also include child * resources. After collecting all the necessary paths, it triggers package * creation. * </p> * * @param resourceResolver The resource resolver used to access the content * repository. * @param pagePaths A comma-separated list of page paths to collect * references from. * @param packageName The desired name of the resulting content package. * @param packageDescription A brief description for the resulting content * package. * @param getChildren A flag indicating whether to include child * resources. Expected values are "YES" or "NO". * @param pageLevel The depth level to which child pages should be * considered. * @return The path to the created package or an empty string if an exception * occurred during package creation. */ private List<String> collectReferences(ResourceResolver resourceResolver, String pagePaths, String packageName, String packageDescription, String getChildren, int pageLevel, int damNodeCount) { Set<String> xfPaths = new HashSet<>(); Set<String> damPaths = new HashSet<>(); ContentVisitor contentVisitor = new ContentVisitor(); Set<String> contentPagePaths = getResourceSet(resourceResolver, pagePaths, pageLevel, getChildren); if (!contentPagePaths.isEmpty()) { iterateContentPaths(resourceResolver, xfPaths, damPaths, contentVisitor, contentPagePaths); return packagePaths(resourceResolver, contentPagePaths, xfPaths, damPaths, packageName, packageDescription, damNodeCount); } return new ArrayList<>(); } /** * Generates a list of package paths based on the provided content paths, XF * paths, DAM paths, package name, package description, and DAM node count. The * DAM paths will be split into multiple packages if their size exceeds the * specified damNodeCount. * * @param resourceResolver the ResourceResolver to resolve resources * @param contentPaths a set of content paths to be included in the * package * @param xfPaths a set of experience fragment paths to be included * in the package * @param damPaths a set of DAM paths to be included in the package * @param packageName the name of the package * @param packageDescription the description of the package * @param damNodeCount the maximum number of DAM nodes allowed in a single * package * @return a list of package paths created based on the provided parameters */ private List<String> packagePaths(ResourceResolver resourceResolver, Set<String> contentPaths, Set<String> xfPaths, Set<String> damPaths, String packageName, String packageDescription, int damNodeCount) { Map<String, String> packageDefinitionProperties = new HashMap<>(); packageDefinitionProperties.put(JcrPackageDefinition.PN_AC_HANDLING, AccessControlHandling.OVERWRITE.toString()); packageDefinitionProperties.put(JcrPackageDefinition.PN_DESCRIPTION, packageDescription); Session session = null; ResourceResolver resolver = null; Resource thumbnailPath; Set<Resource> combinedPaths = null; List<String> packagePaths = new ArrayList<>(); try { thumbnailPath = resourceResolver.getResource(PageConstants.QUERY_PACKAGE_THUMBNAIL_RESOURCE_PATH); final Map<String, Object> authInfo = Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, "demoSystemUser"); resolver = resourceResolverFactory.getServiceResourceResolver(authInfo); session = resolver.adaptTo(Session.class); // Combine content and XF paths combinedPaths = Stream.concat(contentPaths.stream(), xfPaths.stream()).map(resourceResolver::resolve) .collect(Collectors.toSet()); // Create the main package if DAM paths are within limits or empty if (damPaths.isEmpty() || damPaths.size() <= damNodeCount) { combinedPaths.addAll(damPaths.stream().map(resourceResolver::resolve).collect(Collectors.toSet())); packagePaths.addAll(createAndReturnPackagePath(session, thumbnailPath, combinedPaths, packageName, packageDefinitionProperties)); } else { LOGGER.info("1111111111111 QAQAQA#### combinedPaths.size=={}", combinedPaths.size()); LOGGER.info("2222222222222 QAQAQA#### contentPaths.size=={}", contentPaths.size()); LOGGER.info("3333333333333 QAQAQA#### xfPaths.size== {}", xfPaths.size()); LOGGER.info("4444444444444 QAQAQA#### damPaths.size== {}", damPaths.size()); // Create the main package with content and XF paths packagePaths.addAll(createAndReturnPackagePath(session, thumbnailPath, combinedPaths, packageName, packageDefinitionProperties)); int packageCount = 1; for (Set<Resource> damPackageResources : damPaths.stream().map(resourceResolver::resolve) .collect(Collectors.groupingBy(it -> new AtomicInteger().getAndIncrement() / damNodeCount)) .values().stream().map(HashSet::new).collect(Collectors.toList())) { String subPackageName = packageName + "-" + packageCount; packagePaths.addAll(createAndReturnPackagePath(session, thumbnailPath, damPackageResources, subPackageName, packageDefinitionProperties)); packageCount++; } } } catch (Exception e) { LOGGER.error("Exception occured in packagePaths method...", e); } finally { try { // Close the Session if it's not null if (null != session) { session.logout(); // Or the appropriate closing method } } catch (Exception e) { LOGGER.error("Error closing Session: ", e); } finally { try { // Close the ResourceResolver if it's not null if (null != resourceResolver) { resourceResolver.close(); } if (null != resolver) { resolver.close(); } } catch (Exception e) { LOGGER.error("Error closing ResourceResolver: ", e); } } } LOGGER.info("combinedPaths.size=={}", combinedPaths.size()); LOGGER.info("contentPaths.size=={}", contentPaths.size()); LOGGER.info("xfPaths.size== {}", xfPaths.size()); LOGGER.info("damPaths.size== {}", damPaths.size()); return packagePaths; } /** * Resolves a set of page paths to their corresponding resources. Based on the * getChildren flag, it can either directly add the page paths or retrieve the * children of the pages. * * @param resourceResolver The resource resolver to be used for resolving the * page paths. * @param pagePaths String containing newline-separated page paths. * @param pageLevel Depth level for retrieving child pages. * @param getChildren Flag to determine whether to retrieve children of the * pages. * @return Set of resolved resources. */ private @NotNull Set<String> getResourceSet(ResourceResolver resourceResolver, String pagePaths, int pageLevel, String getChildren) { Set<String> childSet = new HashSet<>(); Set<String> pageSet = new HashSet<>(Arrays.asList(StringUtils.split(pagePaths, '\n'))).stream() .map(String::trim).collect(Collectors.toSet()); for (String pagePath : pageSet) { PageManager pageManager = resourceResolver.adaptTo(PageManager.class); if (null != pageManager && null != pageManager.getPage(pagePath) && pageManager.getPage(pagePath).isValid()) { Page startPage = Objects.requireNonNull(pageManager).getPage(pagePath); if (StringUtils.equalsIgnoreCase(PageConstants.CHILD_PAGE_YES, getChildren)) { List<String> childPathList; childSet.add(pagePath); int renderDepth = startPage.getDepth() + pageLevel; childPathList = iteratePageAndChildren(startPage, renderDepth); childSet.addAll(childPathList); } else { childSet.add(pagePath + PageConstants.JCR_CONTENT); } } else { LOGGER.info("{} is not Valid Path.", pagePath); } } return childSet; } /** * Iterates over the children of a given page up to a specified depth. * * @param page The starting page for the iteration. * @param depth The depth up to which the children should be retrieved. * @return List of child page paths up to the specified depth. */ private List<String> iteratePageAndChildren(Page page, int depth) { Iterator<Page> childPageIterator = page.listChildren(); while (childPageIterator.hasNext()) { Page childPage = childPageIterator.next(); int childDepth = childPage.getDepth(); if (childDepth <= depth) { childPagePathList.add(childPage.getPath() + PageConstants.JCR_CONTENT); } iteratePageAndChildren(childPage, depth); } return Collections.unmodifiableList(childPagePathList); } /** * Iterates over a set of paths and processes each path with a content visitor. * * @param resourceResolver The resource resolver to be used for resolving the * paths. * @param xfPaths Set to store XF paths obtained from the content * visitor. * @param damPaths Set to store DAM paths obtained from the content * visitor. * @param contentVisitor The content visitor to process each path. * @param pages Set of page paths to iterate over. */ private void iterateContentPaths(ResourceResolver resourceResolver, Set<String> xfPaths, Set<String> damPaths, ContentVisitor contentVisitor, Set<String> pages) { for (String page : pages) { contentVisitor.accept(resourceResolver.resolve(page)); xfPaths.addAll(contentVisitor.getXfPaths()); // contains experience frament paths damPaths.addAll(contentVisitor.getDamPaths());// contains dam paths } } /** * Creates a package with the given resources, package name, and package * definition properties, and returns the package path in a list. * * @param session The ResourceResolver to be used for * accessing resources * @param thumbnailPath Thumbnail of the package * @param packageResources A set of resources to be included in the * package * @param packageName The name of the package to be created * @param packageDefinitionProperties A map of properties to be added to the * package definition * @return A list containing the path of the created package */ private List<String> createAndReturnPackagePath(Session session, Resource thumbnailPath, Set<@NotNull Resource> packageResources, String packageName, Map<String, String> packageDefinitionProperties) { PackageHelper pckHelper = this.packageHelper; Packaging lPackaging = this.packaging; List<String> packagePathArray = new ArrayList<>(); JcrPackageManager packageManager; JcrPackage jcrPackage = null; try { if (session == null) { return Collections.emptyList(); // Return empty list if session is not available } jcrPackage = pckHelper.createPackage(packageResources, session, PageConstants.DEFAULT_GROUP_NAME, packageName, PageConstants.DEFAULT_VERSION, PackageHelper.ConflictResolution.Replace, packageDefinitionProperties); packageHelper.addThumbnail(jcrPackage, thumbnailPath); packageManager = lPackaging.getPackageManager(session); packageManager.assemble(jcrPackage, null); packagePathArray.add(Objects.requireNonNull(jcrPackage.getNode()).getPath()); } catch (IOException | RepositoryException | PackageException e) { LOGGER.error("Exception occurred during package createAndReturnPackagePath method ", e); } finally { if (null != jcrPackage) { jcrPackage.close(); } } return packagePathArray; } }