Level 9

Solved

AemaaCS - dynamic page generation and robots/sitemap

Forum|Forum|1 year ago
May 16, 2024
4 replies
3601 views

Hi all,

Was hoping to seek suggestions from the group here on thoughts

I need pages dynamically generated from a source and authoring is not an ask , update can happen to the source every min as part of content refresh from source

given this I see an option and noted questions

on loading a page with query param

1. have a single template laid out and replicated

2. use query param and based on it get a response that will be consumed and using sly include include a resource passing independent page component as models as req object to the components for rendering the content

3.- I don’t believe selectors is used these days ? Like a.12344.html and selector 12344 is used to generate content ?
4.- what happens to the OOTB sitemap generator ? It definitely would not show the query param based url in the sitemap since these are dynamic pages . Am I mistaken you think ?

kind regards,

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.

Best answer by Nitin_laad

Hi @nitrohazedev , you should be able to achieve given ask following the below steps -

1. Automated Data Retrieval:

To ensure your content stays up-to-date, consider setting up a scheduled service. This service will run at regular intervals (e.g., hourly, daily) and automatically fetch the latest data from your external source and store it in AEM Repo.
2. Event-Driven Updates ():

User workflow launcher or event listener to trigger workflow or servlet, based on add/Modification of source property in AEM
3. Dynamic Page Management:

Servlet or workflow process will take care of below -
Creating new AEM pages based on the retrieved data.
Updating existing pages to reflect any changes in the data.

EstebanBustamante

Community Advisor and Adobe Champion

Hi,

Sorry, I don’t fully understand your question, but a query string is definitely not the best approach. I suggest creating a servlet that receives a JSON payload containing all the details for page creation. The page creation process should be straightforward. As for the sitemap, it should be updated using a scheduled job to ensure that your new page is added the next time it runs, for example, at midnight.

Hope this helps.

Esteban Bustamante

N

NitroHazeDevAuthor

Level 9

No worries , Why would query param not be the right approach ?
The need is to not store in AEM , no authoring is the ask . Also the content needs a refresh every sec not worth running scheduler on publisher (or author) and persisting nodes or creating pages. Think of it as a single page that displays dynamic data. Imagine components that retrieve dynamic data on a page . Given this the question arises with sitemap

Nitin_laad

Accepted solution

Community Advisor

Hi @nitrohazedev , you should be able to achieve given ask following the below steps -

1. Automated Data Retrieval:

To ensure your content stays up-to-date, consider setting up a scheduled service. This service will run at regular intervals (e.g., hourly, daily) and automatically fetch the latest data from your external source and store it in AEM Repo.
2. Event-Driven Updates ():

User workflow launcher or event listener to trigger workflow or servlet, based on add/Modification of source property in AEM
3. Dynamic Page Management:

Servlet or workflow process will take care of below -
Creating new AEM pages based on the retrieved data.
Updating existing pages to reflect any changes in the data.

N

NitroHazeDevAuthor

Level 9

The need is to not store in AEM , no authoring I the ask . Also the content needs a refresh every sec not worth running scheduler on publisher (or author) and persisting nodes . Besides with running scheduler on pub , not sure what caveats we would run into on cloud .

joerghoh

Adobe Employee

I understand that you have an external source, and AEM needs to deliver the content based on that source. And as that source can receive changes at any time, these changes need to be reflected on the content generated by AEM as well.

A few questions:

Is there any personalization on top of that content source? In other words: Is there only 1 valid version of the content AEM needs to generate at any time?
What is the requirement regarding the latency from "change on source" to "AEM displays this change"? Real-time (that means less than 5 seconds)? 5 minutes? 1 hour?
What's the expected traffic hitting for this content? Total number of requests per day plus number of requests per minute in peak?

With these additional information it's possible to come up with a few ideas and build an application design for it.

N

NitroHazeDevAuthor

Level 9

Thanks @joerghoh as always , please find the responses below. I have used selectors in the past combined with resource type to ensure component design is brought in with dynamic data but pls let me know. The components exist in AEM and I planned to bring in a model on load that would have child model resources like text title etc that I would include on the main page via includes and res type feeding in data via setters for backend.i know AEMaaCS can do it and want to suggest its use

Is there any personalization on top of that content source? In other words: Is there only 1 valid version of the content AEM needs to generate at any time?
- [NZ] No personalization needed , the page might be generated from a search results link with an id as a query param that page will query and display and all of data on it is dynamic . The layout is fixed with placeholders for content that might or might not exist in json that gets fed into it. Now search can provide like 60 80 200 results and some or all of these on click takes to a page that displays content dynamically. There will be analytics tracking and for SEO- robots or sitemap I guess I will have to write a separate sitemap xml to have these query param urls for crawler to crawl and track

What is the requirement regarding the latency from "change on source" to "AEM displays this change"? Real-time (that means less than 5 seconds)? 5 minutes? 1 hour?
[NZ] - Real time is the ask , immediate on load for user . So if a user clicks the search page and json was updated this should be reflected even if it updated as the click occurred .

What's the expected traffic hitting for this content? Total number of requests per day plus number of requests per minute in peak?

[NZ]- I don’t have the answer for this yet since the folks haven’t gotten back to me on this one , but this can be huge , I can get back on this.Since it is a public site so the traffic is assumed to be heavy wrt requests. There appears no limit on api requests. It’s a new page that didn exist and migration to AEM is the ask

joerghoh

Adobe Employee

Thanks @joerghoh reread the post by you and updated below , this is helpful, last thought on this input wrt caching you’ve provided and

yes if clientlibs (not front end app since we might have to write new react based components) is used to retrieve content for both below options - js in AEM client libs , btw SDI is another option perhaps for each comp ?

options

https://www.site.com/page.id.html

https://www.site.com/page.html?#id=1

sitemap Seo

- how would sitemap Seo respond to either ? Custom sitemap with urls hardcoded in robots and have rest generated by Adobe sitemap ?

for the selector , cached copy per id is stored,so any id retrieves data for it
Questions For selector

1. if data at source changes would the cached copy get updated with the fresh response from json?
2. is it a good practice to pass id in selector as a parameter for displaying data ?

for the one with # , if id =123 is triggered first and data for id=123 gets cached at cdn or dispatcher or both

Questions for #

1. Now if a new id =456 is keyed in , the fresh data retrieved by say Ajax for id=456 , would be displayed to the user or would cached version id=123?

2. Again if someone keys in id=123, and if data for id=123 that was originally cached has now changed at the source and someone tried accessing id=123 again , what’s displayed cached data or latest Ajax data is retrieved ?

I believe the latest content retrieved by Ajax is displayed since it would be querying every time page or component loads and is dynamic and cache now contains the latest version if that id is persisted ?

update - added a point to seo original question asked

When you add the topic sitemap and search engines, I think that you might end up with implementing both approaches:

* Use the selector-based approach to render a version for the search engines. It will also include a JS snippet, which switches to the "anchor-based approach" ("#") if a browser visits that URL (and it will silently ignore the values in the page).

* A use the "anchor-based approach" for browsers, which then can directly reach out to the backend.

Then you can reference the selector-based URLs in the sitemap. Depending on the capabilities of the search engine crawler it either behaves like a browser and fetches live data as well from the backend, or it will fetch the embedded values (as determined by the selector).

For the request of your question: I don't get what my mean with the different ID values, switching between them and what to display to the user.

When you use the anchor-based approach ("#"), AEM will never know what kind of value you will display, and for what reason it only render the frame of the page. It's then the task of the Javascript in the browser to obtain these values from the backend and display them within the page. No page with values is cached anywhere.

kautuk_sahni

Community Manager

@nitrohazedev Did you find the suggestions from users helpful? Please let us know if more information is required. Otherwise, please mark the answer as correct for posterity. If you have found out solution yourself, please share it with the community.

Kautuk Sahni

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded