Access Cloud Manager environment variables into Docker

[mmarascu]

I’m trying to make some configurable docker keys from Cloud Manager Environment.

How can I pass environment configurations (secrets) from Cloud Manager to Docker file?

[Anmol_Bhardwaj]

You can download the environment variables as a JSON file from the API provided by adobe:

https://cloudmanager.adobe.io/api/program/{programId}/environment/{environmentId}/variables

After that, you can easily store them in Docker.

Reference: https://developer.adobe.com/experience-cloud/cloud-manager/reference/api/#operation/getEnvironmentVa... 

[miklosc53670490]

I think the question is more about how to NOT need to hardcode them in DockerFile. 
I don't want to hardcode them in the code and push them to the git repository. 

Can we access the above API from the docker instance that is created with the selenium tests in Cloud Manager? Without needing to place a "SECRET_KEY" inside the docker instance or anywhere in the code. It should come from the Cloud Manager configurations (environment) 

[mmarascu]

So what I need is to access those secret variables while running the ui.test docker container in Cloud Manager pipeline.

[Jörg_Hoh]

Can you please give some context what you mean with "Docker file"? Where in the context of Cloud Manager do you use a Docker file?

[mmarascu]

I will try to be more explicit, so I am not able to read the configured Cloud Manager environment variable  when doing the docker-compose up in the Custom UI testing step. I have tried different places to access them:

- in the pom.xml file of the ui.tests (ui-tests-docker-execution profile - docker compose - environmentVariables <VARIABLE_NAME>${env.VARIABLE_NAME}</VARIABLE_NAME>)

- in the docker compose yaml file as environment

- in the dockerfile as ENV

[Jörg_Hoh]

I am referring to the official documentation.

I understand it in a way, that it is the customer's responsibility to create the container; and that CM will startup the container. It's not that your maven build will start it up directly using some plugin or so.

For the environment variables it is stated "The following environment variables will be passed to your Docker image at run time."

That means that CM will inject these environment variables in your runtime, so that the code there can read them.

Now re-reading your post I assume that you want to run the same container for testing purposes on your local machine. Is this correct?

In that case you need to provide them as environment variables to your command (see the docker documentation, or the docs for docker compose). 

[mmarascu]

Following this official documentation, I should be able to set some secret environment variables in CM and access them at runtime by adding them in the pom.xml file (docker compose profile) under the existing ones.

Simulating this on local host by exporting these variables and then building and running the docker image would actually list them.

Trying the same in CM, would not list these already configured env vars that should be available at runtime in that pipeline. 

[Andres_Peréz]

Did you manage to fix it or any news in this? We're having a similar problem. Tried setting an env var via CLI commands unfortunately build profile is not pulling it.

[JosueZu]

Hey @mmarascu, have you managed to solve this issue? I'm facing pretty much the same issue right now.

[JuliuszP1]

Hello @mmarascu, have you found any solution finally?
I'm facing the same problem trying to setup Playwright ui.tests with private npm repository.

Setting up the tests is not a problem but the only problem is how to pass pipeline variables into the Dockerfile.

My assumption was that those variables/secrets will be visible in pom.xml as env variables, then I will pass them via --build-arg into docker:

<argument>build</argument>
<argument>--build-arg</argument>
<argument>REGISTRY_TOKEN=${REGISTRY_TOKEN}</argument>

When it is run locally it works perfectly:

mvn clean package -Pui-tests-docker-build -DREGISTRY_TOKEN=<BASE64 PAT to registry>

but somehow it doesn't work on real pipeline.

What is more interesting, when I put a fixed value in pom.xml like:

<argument>--build-arg</argument>
<argument>TEST_3=fixedValue</argument>

they are not visible neither. It looks like building the image in step "Build Images" is performed somehow different than "mvn clean package -Pui-tests-docker-build" command (recommended locally).

[JuliuszP1]

I found why my solution didn't work in the cloud.
Looks like the maven profiles in pom.xml: "ui-tests-docker-build" and "ui-tests-docker-execution" are used ONLY in local execution and not in the cloud.

Therefore any changes in build profile (especially adding --build-arg) have no impact on the cloud build process.

In the cloud, in real pipeline it works as follows:
- the archive (tar.gz) with all files necessary to build docker image (incl. Dockerfile and other) is created by Maven Assembly Plugin - this we can control in pom.xml
- later on, some process in the pipeline takes this archive (Docker context) and makes the docker image

after documentation: "The archive containing the Docker build context is automatically picked up by Cloud Manager, which will build the Docker image containing your tests during its deployment pipelines"

Unfortunately, we have no control over this process. And pipeline variables (nor secrets) are not passed to this process at all.

PS. Temporarily, I've made a workaround by shifting the functionality that requires the pipeline secrets (in my case: "npm install") from image building stage (Dockerfile) to runtime stage (run.sh started by ENTRYPOINT).
In the runtime all variables and secrets are passed to docker container correctly.
I know, this way is not a solution of the original problem/question and have many disadvantages, but one advantage: it works.