Level 9

Solved

AemaaCS - dynamic page generation and robots/sitemap

Forum|Forum|1 year ago
May 16, 2024
4 replies
3576 views

Hi all,

Was hoping to seek suggestions from the group here on thoughts

I need pages dynamically generated from a source and authoring is not an ask , update can happen to the source every min as part of content refresh from source

given this I see an option and noted questions

on loading a page with query param

1. have a single template laid out and replicated

2. use query param and based on it get a response that will be consumed and using sly include include a resource passing independent page component as models as req object to the components for rendering the content

3.- I don’t believe selectors is used these days ? Like a.12344.html and selector 12344 is used to generate content ?
4.- what happens to the OOTB sitemap generator ? It definitely would not show the query param based url in the sitemap since these are dynamic pages . Am I mistaken you think ?

kind regards,

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by Nitin_laad

Hi @nitrohazedev , you should be able to achieve given ask following the below steps -

1. Automated Data Retrieval:

To ensure your content stays up-to-date, consider setting up a scheduled service. This service will run at regular intervals (e.g., hourly, daily) and automatically fetch the latest data from your external source and store it in AEM Repo.
2. Event-Driven Updates ():

User workflow launcher or event listener to trigger workflow or servlet, based on add/Modification of source property in AEM
3. Dynamic Page Management:

Servlet or workflow process will take care of below -
Creating new AEM pages based on the retrieved data.
Updating existing pages to reflect any changes in the data.

EstebanBustamante

Community Advisor and Adobe Champion

Hi,

Sorry, I don’t fully understand your question, but a query string is definitely not the best approach. I suggest creating a servlet that receives a JSON payload containing all the details for page creation. The page creation process should be straightforward. As for the sitemap, it should be updated using a scheduled job to ensure that your new page is added the next time it runs, for example, at midnight.

Hope this helps.

Esteban Bustamante

N

NitroHazeDevAuthor

Level 9

No worries , Why would query param not be the right approach ?
The need is to not store in AEM , no authoring is the ask . Also the content needs a refresh every sec not worth running scheduler on publisher (or author) and persisting nodes or creating pages. Think of it as a single page that displays dynamic data. Imagine components that retrieve dynamic data on a page . Given this the question arises with sitemap

Nitin_laad

Accepted solution

Community Advisor

Hi @nitrohazedev , you should be able to achieve given ask following the below steps -

1. Automated Data Retrieval:

To ensure your content stays up-to-date, consider setting up a scheduled service. This service will run at regular intervals (e.g., hourly, daily) and automatically fetch the latest data from your external source and store it in AEM Repo.
2. Event-Driven Updates ():

User workflow launcher or event listener to trigger workflow or servlet, based on add/Modification of source property in AEM
3. Dynamic Page Management:

Servlet or workflow process will take care of below -
Creating new AEM pages based on the retrieved data.
Updating existing pages to reflect any changes in the data.

N

NitroHazeDevAuthor

Level 9

The need is to not store in AEM , no authoring I the ask . Also the content needs a refresh every sec not worth running scheduler on publisher (or author) and persisting nodes . Besides with running scheduler on pub , not sure what caveats we would run into on cloud .

joerghoh

Adobe Employee

I understand that you have an external source, and AEM needs to deliver the content based on that source. And as that source can receive changes at any time, these changes need to be reflected on the content generated by AEM as well.

A few questions:

Is there any personalization on top of that content source? In other words: Is there only 1 valid version of the content AEM needs to generate at any time?
What is the requirement regarding the latency from "change on source" to "AEM displays this change"? Real-time (that means less than 5 seconds)? 5 minutes? 1 hour?
What's the expected traffic hitting for this content? Total number of requests per day plus number of requests per minute in peak?

With these additional information it's possible to come up with a few ideas and build an application design for it.

N

NitroHazeDevAuthor

Level 9

Thanks @joerghoh as always , please find the responses below. I have used selectors in the past combined with resource type to ensure component design is brought in with dynamic data but pls let me know. The components exist in AEM and I planned to bring in a model on load that would have child model resources like text title etc that I would include on the main page via includes and res type feeding in data via setters for backend.i know AEMaaCS can do it and want to suggest its use

Is there any personalization on top of that content source? In other words: Is there only 1 valid version of the content AEM needs to generate at any time?
- [NZ] No personalization needed , the page might be generated from a search results link with an id as a query param that page will query and display and all of data on it is dynamic . The layout is fixed with placeholders for content that might or might not exist in json that gets fed into it. Now search can provide like 60 80 200 results and some or all of these on click takes to a page that displays content dynamically. There will be analytics tracking and for SEO- robots or sitemap I guess I will have to write a separate sitemap xml to have these query param urls for crawler to crawl and track

What is the requirement regarding the latency from "change on source" to "AEM displays this change"? Real-time (that means less than 5 seconds)? 5 minutes? 1 hour?
[NZ] - Real time is the ask , immediate on load for user . So if a user clicks the search page and json was updated this should be reflected even if it updated as the click occurred .

What's the expected traffic hitting for this content? Total number of requests per day plus number of requests per minute in peak?

[NZ]- I don’t have the answer for this yet since the folks haven’t gotten back to me on this one , but this can be huge , I can get back on this.Since it is a public site so the traffic is assumed to be heavy wrt requests. There appears no limit on api requests. It’s a new page that didn exist and migration to AEM is the ask

joerghoh

Adobe Employee

Thanks.

Having everything public definitely makes things easier.

Using a selector instead of a query string is definitely good to have, it gives you at least the chance to cache the resulting page.

The requirement that changes must be updated and then delivered in real-time to a potentially very large audience is definitely challenging. Because if I implement in a naive way, AEM is just the proxy, and then every enduser-request creates a direct request to your backend system. That's definitely not a good idea, and to reduce the number of backend calls you need to implement a caching layer to reduce these requests.

On the other hand side, can't you implement this using a frontend application, deliver the static frame via AEM and then have Javascript which is fetching this data directly from the backend and embedding it live? Using AEM as a proxy for a potentially very large amount of requests is something I would avoid as good as I can.

kautuk_sahni

Community Manager

@nitrohazedev Did you find the suggestions from users helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.

Kautuk Sahni

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded