Custom UI Testing Tests Not Executing Successfully in Cloud Environment Despite Local Success

[Saumya]

We are encountering an issue where our test suite executes successfully in the local environment but fails to run properly in the cloud environment. Below are the relevant details and observed behavior for troubleshooting:

Local Execution:

Environment: macOS
Logs Summary: 

Ran mvn verify -Pui-tests-local-execution \
-DAEM_AUTHOR_URL=http://localhost:4502 \
-DAEM_AUTHOR_USERNAME=admin \
-DAEM_AUTHOR_PASSWORD=admin \
-DAEM_PUBLISH_URL=http://localhost:4503 \
-DAEM_PUBLISH_USERNAME=admin \
-DAEM_PUBLISH_PASSWORD=admin on local repo

All tests pass as expected.
Sample log output indicates successful execution:
3 passing (1.6s) for /specs/aem/login.js
1 passing (2.5s) for /specs/aem/basic.js
HTML report generated successfully.
Build completes with: BUILD SUCCESS.

Cloud Execution:
Tests do not execute correctly.

Attached dev cloud pipeline logs.

Has anyone faced a similar issue with AEM test execution? How can this be fixed ?

[konstantyn_diachenko]

Hi @Saumya,

As far as I know, Cloud Manager executes UI tests in the Docker, not directly via npm. 

More documentation about different Test engines (Cypress, Selenium etc) you can find here: https://experienceleague.adobe.com/en/docs/experience-manager-cloud-service/content/implementing/usi... . Read about proxy setup. Perhaps it could be a problem. I'd suggest to increase a logging level for the UI tests execution. 

Also, I see that you have errors during the test execution:

[chrome-headless-shell 127.0.6533.72 linux #0-2] 1) AEM Login Page "before each" hook for AEM Login Page
[chrome-headless-shell 127.0.6533.72 linux #0-2] waitUntil condition timed out after 60000ms
[chrome-headless-shell 127.0.6533.72 linux #0-2] Error: waitUntil condition timed out after 60000ms
[chrome-headless-shell 127.0.6533.72 linux #0-2]     at async Browser.AEMForceLogout (file:///usr/src/app/lib/wdio.commands.js:53:5)


node:internal/modules/cjs/loader:1228
  throw err;
  ^

Error: Cannot find module '/usr/src/app/lib/testreportpath.js'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
    at Module._load (node:internal/modules/cjs/loader:1051:27)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)
    at node:internal/main/run_main_module:28:49 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

Seems like your test can't open login page. 

Best regards,

Kostiantyn Diachenko.

[kautuk_sahni]

@Saumya Did you find the suggestions helpful? Please let us know if you require more information. Otherwise, please mark the answer as correct for posterity. If you've discovered a solution yourself, we would appreciate it if you could share it with the community. Thank you!

[Saumya · Accepted solution]

Even after adding the testreportpath.js file, the tests continued to fail on the cloud pipeline, despite executing successfully both local env and on the cloud environment when run locally. To resolve this, we transitioned to using the Cypress framework instead of Selenium, and the Custom UI Testing step is now successfully passing in the cloud pipeline.

Additionally, as per Adobe's documentation, Cypress is the recommended framework for UI testing. However, the AEM Cloud project repo comes with Selenium test cases by default, requiring the entire ui.tests folder to be converted to Cypress for compatibility.

[EstebanBustamante]

Hi @Saumya just for clarification, since version 43 of the AEM archetype, cypress is the default testing framework instead of selenium, you maybe using an older version of the archetype

[Saumya]

Oh okay. Thanks for the info @EstebanBustamante