How to Enable ES6 Javascript Support for Rendering Dynamic AEM Pages Using HtmlUnit?

[arana1]

Hi everyone,

We have a use case where AEM content pages are dynamically populated with content using ReactJS. To fetch the feully rendered page content(including dynamically inserted elements), we rely on org.htmlunit.webclient by enabling the javascript support in our code. This approach worked well earlier, however, after recent updates to our React codebase, we started encountering issues with ES6 features not being supported by htmlunits current js engine (Rhino).
Specifically we are seeing below error: 

org. htmlunit. ScriptException: identifier is a reserved word: class (https://sample-site:8443/etc.clientlibs/test-project/clientlibs/cms-commons-react.js#241)

This indicates that Rhino engine lacks support for modern JS syntax probably with ES6+ features.

We are exploring alternative solutions that allow us to continue rendering full HTML content in our servlets:

1. Is it possible to integrate a more modern JS engine like GraalVM with htmlunit which might resolve the issue we are facing? If yes, how can we set it up to work with latest version of HtmlUnit.

2. Are there other reliable libraries or strategies available for rendering fully dynamic, JS- heavey pages from AEM?

We need a solution that can be used server-side to extract the rendered content. Any guidance for this use case would be highly appreciated! Thank you!

Note- We are on AEM 6.5.19.0
@kautuk_sahni @arunpatidar @kiran5 @lukasz-m @BrianKasingli 

[AmitVishwakarma · Accepted solution]

To enable ES6 support for dynamic AEM pages using HtmlUnit:

1. Upgrade the JavaScript Engine: Integrate a modern engine like GraalVM or use Nashorn for ES6 support. You would need to configure HtmlUnit to use these engines, although HtmlUnit does not natively support GraalVM.

2. Alternative Libraries: Consider using Puppeteer or Selenium WebDriver, which provide full support for modern JavaScript (ES6+) and handle dynamic content rendering with headless browsers.

3. Headless Browsers: Puppeteer (for Node.js) or Selenium (using headless Chrome/Firefox) are more robust options for rendering JS-heavy pages with modern syntax support.

[arana1]

Thank you for your inputs on my original post regarding the issue with HTMLUnit not capturing the full markup from AEM pages that include client-side rendered React content.

I was able to resolve the issue by switching from HTMLUnit (which does not support dynamic JavaScript rendering well) to Puppeteer, a Node.js library that controls headless Chrome.

I wrote a Puppeteer script (render-page.js) that:

• Launches a headless Chrome browser
• Navigates to the AEM page URL
• Waits for the page to fully load and React content to render
• Extracts the final HTML content

I created an AEM servlet (Java) that:

• Constructs the page URL to fetch
• Calls the Puppeteer script using a ProcessBuilder
• Captures the output (fully rendered HTML)
• Returns it as the servlet response

[kautuk_sahni]

@arana1 Did you find the suggestion helpful? Please let us know if you need more information. If a response worked, kindly mark it as correct for posterity; alternatively, if you found a solution yourself, we’d appreciate it if you could share it with the community. Thank you!

[arana1]

Thank you. I did find the response useful. One of the suggestions worked for me, have updated how I was able to resolve it.

[luismariano9001]

I have a similar problem. If you solved your problem let us know "How to do". Thanks.

[arana1]

I wasnt able to get HTMLUnit worked. Tried to make it use more modern JS engine but that relied on too many different dependencies ultimately causing the bundle to not resolve. So I went ahead with a different solution using Puppeteer script as an API. Hosted this as a standalone API on one of our server and requested the same through AEM servlet. Hope this works for you.