AEM CIF + Magento on Publish

[evancooperman-r]

Hi everyone,

I'm struggling a bit to get my publish instance working properly with AEM + CIF (1.10.0) + Venia + Magento.  Everything works properly in author - I can view products, add to cart, and complete checkout.  My commerce config path is set as /api/graphql.

The problem is that in publish, I'm getting a 403 error when it attempts to hit the graphql endpoint:

http://localhost:4503/api/graphql

I have set up the COMMERCE_ENDPOINT environment variable and this is working as expected in author.

I checked the CSRF configuration and by default it only has the /graphql endpoint listed as ignoring CSRF, so I'm assuming that the publish endpoint cannot just be /api/graphql.  I tried adding /api/graphql to the CSRF config just for fun, but that didn't fix it. 

So my question is, in order for publish to work, do I have to set up a proxy and update my config in author (and then publish it) using that URL instead?  I actually tried changing the URL to hit Magento directly, but of course I get a CORS error, so there obviously needs to be a proxy somewhere...I just thought AEM could handle it internally for publish like it does for author.  Does anyone know if I absolutely need to set up a proxy for publish?  Even if I do use a proxy, I'm assuming I would want to use the dispatcher as my proxy so that I don't run into the CORS issue there too, and to avoid a second proxy, right?

I've tried scouring the docs but they all speak primarily to author, with almost nothing at all for how the publish setup differs.  The only thing I see is the following, which seems to indicate that AEM publish should function as it's own proxy the same way author does:

Example Mac OSX:

export COMMERCE_ENDPOINT=https://demo.magentosite.cloud/graphql

Example Windows:

set COMMERCE_ENDPOINT=https://demo.magentosite.cloud/graphql

This variable is used by AEM to connect to your commerce system. Also, the CIF add-on includes a local reverse proxy make the Commerce GraphQL endpoint available locally. This is used by the CIF authoring tools (product console and pickers) and for the CIF client-side components doing direct GraphQL calls.

Thanks in advance!

[moek18753296]

Are you logging in on the publisher first?  If not, I would check the Sling Authentication Service in the OSGi console is set to allow Anonymous access. https://experienceleaguecommunities.adobe.com/t5/adobe-experience-manager/aem-sites-locked-for-anony...

[evancooperman-r]

Great thought, and for a minute I thought you were on to something! Unfortunately my Sling Authentication Service is already allowing anonymous access to the repo. I also tested again after logging in as admin, but getting the same issue still anyway.

[shelly-goel]

@evancooperman-r  I hope you've activated your /conf/venia cloud configurations

[evancooperman-r]

I did activate them, the URL it's attempting to hit in publish is the same as in author (but port 4503), but I'm getting the 403 error.

[shelly-goel]

@evancooperman-r  Yes the graphql endpoint should be /api/graphql in publish as well. Where do you get 403?

Is your issue resolved?

[Aryan12345]

Same issue here as well (in 2022 :))

Did you find any solution?

This is the summary of my issue

The /api/graphql is throwing 403 errors which means no frontend commerce operation is happening on the storefront.

I tried adding the /api/graphql to the CSRF filter's exclude list and even removed the POST method from there.

After this I still 403 on GET request of /api/graphql and the following on POST request.

I am I missing some setting?

Note: on the sling servlet resolver config, I have added the /api/graphql but there is no change.

[evancooperman-rp]

I updated my local dispatcher tools to have an ENVIRONMENT_LOCAL option, and then use that in conjunction with an addition I made to our vhost file like this:

<IfDefine ENVIRONMENT_LOCAL>
    ProxyErrorOverride off
    ProxyPass         /api/graphql http://host.docker.internal:3001
    ProxyPassReverse  /api/graphql http://host.docker.internal:3001
</IfDefine>

 so I just use the dispatcher as a proxy.  Since the dispatcher starts in a VM, I believe you also need a local npx proxy in order to forward traffic along to Adobe I/O like this:

npx local-cors-proxy --proxyUrl https://adobeioruntime.net/api/v1/web/xxxx/graphql-reference/dispatcher --port 3001 --proxyPartial ""

Hope that helps!

[Aryan12345]

Super... thats cool... thanks for sharing.

Just today I fixed it in a slightly different (and a lazy way). I introduced a var called COMMERCE in the global vars since the vhost file was referring to the same:

# CQ-4287185: Allow access to magento reverse-proxy endpoint
<IfDefine COMMERCE>
    SSLProxyEngine on
    <LocationMatch "/api/graphql">
        ProxyPass        ${COMMERCE_ENDPOINT}
        ProxyPassReverse ${COMMERCE_ENDPOINT}
        RewriteEngine Off
    </LocationMatch>
</IfDefine>

Could have done it better, but as I said, i guess I am lazy.

One question though: Why does it work in author without all this magic and not in publish? What makes publish special?

[evancooperman-rp]

This is the right way to do it, I forgot the CIF team had added this into the codebase now.  I haven't been working on the CIF for a minute, but now I remember that they added this code in.  I should probably upgrade ours to use the CIF version of the reverse proxy rather than our custom one

It works in author because they built a proxy into the author instance via code, but the proxy is not active in publish.  My assumption is that they did that to ensure that all the authoring pieces will work (dialogs, commerce screen, etc.) without having to configure anything special.  But, from a performance standpoint, the real proxy is better since it prevents requests from actually hitting AEM at all in a publish scenario.

[Yonit]

This is for the sdk?  Adding this in global.vars in dispatcher fixed a problem on localhost:4503? I tried this and it did not work.

I get Forbidden for localhost:4503/api/graphql and 404 for localhost:8080/api/graphql

What do you mean that your vhost was referring to a COMMERCE variable?  in what way?

Thanks

Yonit

[evancooperman-rp]

No you need to add a proxy in your dispatcher that does this.  Look at the code he posted above, that is included in the CIF:

# CQ-4287185: Allow access to magento reverse-proxy endpoint
<IfDefine COMMERCE>
    SSLProxyEngine on
    <LocationMatch "/api/graphql">
        ProxyPass        ${COMMERCE_ENDPOINT}
        ProxyPassReverse ${COMMERCE_ENDPOINT}
        RewriteEngine Off
    </LocationMatch>
</IfDefine>

you'll need to make sure the COMMERCE variable is set to true, and then it should enable your proxy within the dispatcher, assuming you have the latest version of the dispatcher code (you might have to pull the latest and merge it into your codebase if you have an older version).  Whatever port you're running the dispatcher, on, you should be able to hit /api/graphql and it should forward these requests, via dispatcher proxy, to the commerce system.

See https://github.com/adobe/aem-project-archetype/blob/d1d62d464e9bbf28d2379d043aed5f90adc48aab/src/mai... for the file in the archetype that has the proxy configuration in it.

[Yonit]

I was able to resolve this for my local dispatcher by doing the proxy to the magento cloud graphql endpoint, but I cannot proxy my local magento installation.  Also, the dispatcher doesn't help with local publish.  Adobe CIF GraphQL Proxy Configuration does not exist on publish so can't work like it does on author.

[evancooperman-rp]

You should be able to proxy your local Magento installation the same way you would proxy to the cloud instance.  The Adobe CIF GraphQL Proxy Configuration doesn't exist in publish which is exactly why you need to use your dispatcher as the proxy instead of relying on the proxy that ships with AEM Author.  You need your dispatcher to pick up all requests to localhost:8080/api/graphql (or whatever port you have the dispatcher running on) and proxy them directly to the commerce system, thus bypassing AEM altogether.

I thought I'd responded to this last year, but I guess not. I hope you figured out how to resolve your issue by now...or if not, I hope this helps!

[Gokula_Krishnan_GL]

HI ,
We are running a similar implementation where Commerce system is SAP Hybris by using CIF implementation and IO layer to run graphql queries. Currently am trying to proxy the graphql endpoint for which i followed the above steps in dispatcher vhost file, however i'm getting 503 service unavailable error .

I have defined COMMERCE to be as true in custom vars and ${COMMERCE_ENDPOINT} in system variable with actual endpoint of the IO runtime. 
I have also referred other reference and used like below, but no luck.
<IfDefine COMMERCE>
SSLProxyEngine on
#  add ProxyRemote to tunnel reverse-proxy traffic through egress proxy if available
<IfDefine HTTP_EGRESS_PROXY>
ProxyRemote ${COMMERCE_ENDPOINT} "http://host.docker.internal:8080"
</IfDefine>
<LocationMatch "/api/graphql(/custom)?$">
# Use an empty back reference from ProxyPassMatch to the LocationMatch regex to prevent the
# original URL being appended to the proxy request
ProxyPassMatch ${COMMERCE_ENDPOINT}$2
ProxyPassReverse ${COMMERCE_ENDPOINT}
RewriteEngine Off
</LocationMatch>
</IfDefine>

can someone help me here as it is critical to solve the issue and point me with right steps to achieve this .

Regards,

Gokul
@Aryan12345 @evancooperman-rp @Yonit 

[evancooperman-rp]

So you get a 503 error when going through the dispatcher, but a 200 when you go directly to Adobe I/O?  I think I'll need a little more info to help.  My use-case from this original post was using a third-party commerce system through Adobe I/O as well, so I know it's possible.  In that case, I remember that Adobe I/O was running over https, so I had to have an additional proxy that just converted traffic from https to http using a self-signed certificate locally, in order to make browser traffic go over SSL, but still hit AEM over http.  I used an npm module called local-ssl-proxy to enable SSL in my local system without having to configure it in the dispatcher, with the command:

local-ssl-proxy --source 443 --target 80

(my dispatcher was running on port 80).   This effectively allowed me to make https calls via the browser, and have the browser treat them as SSL calls, but then terminate SSL before sending requests to AEM to avoid issues on that side.  I hope that helps, but might need a bit more info to help further.

[Gokula_Krishnan_GL]

HI @evancooperman-rp 

There are two things i wanted to achieve.
 1. We wanted to make the IO endpoint which appears in head tag to be a friendly url (proxy path ) , which will be consumed by FE for clientside calls. we have enabled the 'enable proxy path' in CIF cloud services 

2. In continuation to above, we wanted to achieve dynamic price load ( enabled dynamic pricing checkbox in cloud config)

I set the below change in vhost file and have few doubts here. 
Fr cloud instances, COMMERCE_ENDPOINT will be a env variable in CM and in local it will be in system env variables?

what needs to be passed for "http://${AEM_HTTP_PROXY_HOST}:${AEM_HTTP_PROXY_PORT}" ( for both local and Cloud instances )

<IfDefine COMMERCE>
SSLProxyEngine on
# CIF-2557 add ProxyRemote to tunnel reverse-proxy traffic through egress proxy if available
<IfDefine HTTP_EGRESS_PROXY>
ProxyRemote ${COMMERCE_ENDPOINT} "http://${AEM_HTTP_PROXY_HOST}:${AEM_HTTP_PROXY_PORT}"
</IfDefine>
<LocationMatch "/api/graphql(/default)?$">
# Use an empty back reference from ProxyPassMatch to the LocationMatch regex to prevent the
# original URL being appended to the proxy request
ProxyPassMatch ${COMMERCE_ENDPOINT}$2
ProxyPassReverse ${COMMERCE_ENDPOINT}
RewriteEngine Off
</LocationMatch>
</IfDefine>

Kindly suggest on sample values as i continue to get 503 or 403 forbidden error for the api/graphql/default endpoint. I have referred to above comments and tried all possible things but not succeeded , may be some syntax error or a step is missed.

Thanks,

Gokul

[evancooperman-rp]

The proxy path in AEM has no bearing on publish.  Proxy path is purely for author.  AEM author ships with an internal proxy (via code) that is not available on the publisher.  What you need to ensure is that your dispatcher proxy is set up and pointing to Adobe I/O, and that should be that.  Can you confirm that when you make calls, via something like Postman, directly to Adobe I/O, that it responds as expected? 

When I did this, I would start the dispatcher with a command like

DISP_RUN_MODE=local ./docker_run.sh out host.docker.internal:4503 80

which would then go into my IfDefine block here (I had to update some core files inside of dispatcher-tools in order to make this work, but you should be able to use the OOTB configs to do the same thing):

<IfDefine ENVIRONMENT_LOCAL>
    ProxyErrorOverride off
    ProxyPass         /api/graphql http://host.docker.internal:3001
    ProxyPassReverse  /api/graphql http://host.docker.internal:3001
</IfDefine>

and would set up my proxy.  I don't know if setting a normal environment variable will suffice, so maybe make sure you're running something like 

COMMERCE_ENDPOINT=<Adobe_IO_Endpoint> ./docker_run.sh out host.docker.internal:4503 80

For me when I was doing it with my configs above, I had set up another local proxy that passes everything from port 3001 to Adobe I/O - that's the proxy that actually gets the dispatcher to properly forward along to Adobe I/O:

npx local-cors-proxy --proxyUrl https://adobeioruntime.net/api/v1/web/544668-trexprime-dev/graphql-reference/dispatcher --port 3001 --proxyPartial ""

but it's possible you can just supply that endpoint in your start command and do away with the local-cors-proxy.  I'm trying to remember why I did it this way, but I may have just been making things harder than they needed to be.

Passing variables into the Dispatcher is shown here https://experienceleague.adobe.com/docs/experience-manager-learn/cloud-service/local-development-env... in the Dispatcher Tools Logs section.

Hope that helps!

[rajeevyadav1]

We are not using Adobe Commerce and are looking to use CIF with an integration layer like Mulesoft. This integration layer will provide the graphql data by connecting to other commerce/PIM system.

Can the CIF Magento endpoint service support if integration layer requires authentication mechanism? In Cloud configuation, it has option for qraphql endpoint, store name and uid, however don't see any option for authentication with endpoint. Does this mean that graphql endpoint need to be public?