Custom assets compute worker

[Marcin_Czeczko]

Hi. I found out that I can init firefly app for assets compute workers. So I did created firefly project in io console having assets-compute APIs added. Then I was able to init my new project using AIO CLI that generated for me simple worker that simply creates rendition by copying source asset.

Local tests went fine. I run worker locally thanks to the aio-cli-assets-compute, i can run assets compute devtool. However, I couldn't test my worker wither locally or once deployed to io runtime. 

Does this suppose to work already or its something that's not suppose to be available officialy yet ?

I can share more details (logs, etc) if required.

[Marcin_Czeczko]

I digged into the issue (I was able to enable debug logging for shared validators used in the actions sequence for my worker) and my findings are:

- The asset-compute service invocation from AssetCompute Devtool works fine

- The service calls my worker, but the /adobeio/shared-validators-v1/headless throws 403 as my input IMS Org Id doesn't match to something - seems that it tries to match to string "true", see the log I captured. I obfuscated sensitive data and my actual organization. Most important is the line that it's a little bit odd: "input IMS org id '<MY_IMS_ORG>@AdobeOrg' does not match app IMS org id 'true'" (of course it consists my ims org) 

2020-08-06T07:10:36.324Z       stdout: 2020-08-06T07:10:36.324Z [main /adobeio/shared-validators-v1/headless] info: Calling the main action
2020-08-06T07:10:36.325Z       stdout: 2020-08-06T07:10:36.325Z [main /adobeio/shared-validators-v1/headless] debug: params: {
   "source":{
      "mimetype":"application/octet-stream",
      "name":"20200514_085516.jpg",
      "size":1671305,
      "url":"https://XXXXXXXXX.blob.core.windows.net/firefly/source/20200514_085516.jpg?sv=2019-02-02&spr=https&se=2020-08-06T09%3A10%3A35Z&sr=b&sp=r&sig=XXXXXXXXXXX"
   },
   "__ow_method":"post",
   "userData":{
      "assetComputeClient":{
         "id":"74b0c0c0-d7b3-11ea-8b8c-df7dcfc540c3"
      }
   },
   "newRelicEventsURL":"https://insights-collector.newrelic.com/v1/accounts/2045220/events",
   "__ow_headers":{
      "accept":"*/*",
      "accept-encoding":"gzip, deflate",
      "authorization":"<hidden>",
      "connection":"close",
      "content-type":"application/json",
      "host":"controller-a",
      "perf-br-req-in":"1596697835.969",
      "user-agent":"node-fetch/1.0 (+https://github.com/bitinn/node-fetch)",
      "x-app-name":"Project_1596623723209J1",
      "x-forwarded-for":"X.X.X.X, X.X.X.X",
      "x-forwarded-host":"XXXXXXXXX-dev.adobeioruntime.net",
      "x-forwarded-port":"443",
      "x-forwarded-proto":"https",
      "x-gw-ims-org-id":"<MY_IMS_ORG>@AdobeOrg",
      "x-gw-ims-org-name":"<MY ORG NAME> Sandbox",
      "x-real-ip":"10.250.204.164",
      "x-request-id":"9zA6pY2h61iFaS9cmL8NGyd0m5Or2Ja7"
   },
   "times":{
      "core":"2020-08-06T07:10:35.849Z",
      "gateway":"2020-08-06T07:10:35.000Z",
      "process":"2020-08-06T07:10:35.817Z"
   },
   "__ow_path":"",
   "renditions":[
      {
         "name":"rendition.jpg",
         "target":"https://XXXXXX.blob.core.windows.net/firefly/rendition/20200514_085516.jpg/XXXXXXXXXX/0/rendition.jpg?sv=2019-02-02&spr=https&se=2020-08-06T09%3A10%3A35Z&sr=b&sp=cw&sig=XXXXXXXX&comp=block&blockid=XXXXXX",
         "userData":{
            "assetComputeClient":{
               "index":0,
               "length":1
            },
            "path":"rendition/20200514_085516.jpg/XXXXXXXX/0/rendition.jpg"
         },
         "worker":"https://XXXXXXXX-dev.adobeioruntime.net/api/v1/web/AEMMczAssetCompute-0.0.1/mcz-aem-asset-computer-worker?__VALIDATOR_LOG_LEVEL=debug"
      }
   ],
   "newRelicApiKey":"4iUx7ZvvK_PJ-FPLFDizxO3HllDa9LGN"
}
2020-08-06T07:10:38.901Z       stdout: 2020-08-06T07:10:38.901Z [main /adobeio/shared-validators-v1/headless] info: input IMS org id '<MY_IMS_ORG>@AdobeOrg' does not match app IMS org id 'true'
2020-08-06T07:10:38.902Z       stdout: 2020-08-06T07:10:38.902Z [main /adobeio/shared-validators-v1/headless] info: 403: request is invalid, failed authorization. Please use a valid JWT or user access token for this headless application.

[User_Ars]

Hi @Marcin_Czeczko -

How did you manage to enable the debug logs on the shared validators on the sequence, could you please share your inputs?

Thanks!

@duypnguyen @sarahxxu 

[Marcin_Czeczko]

I diged into the issue (I managed to enable debug logging for shared validators used in the worker sequence) and my findings are:

- Asset compute service is called fine

- Asset compute service calls my worker and it fails with 403

- The reason is that the /adobeio/shared-validators-v1/headless used in my worker sequence throws 403 response code as for some reason my input IMS Org ID doesn't match with the string literal "true" which is quite odd.

See the logs from the validators action I captured. I obfuscated the sensitive data but you can easily spot the last two lines that shows the whole processing stops in here. Does anyone have any cloud what's going on ?

2020-08-06T07:10:36.324Z       stdout: 2020-08-06T07:10:36.324Z [main /adobeio/shared-validators-v1/headless] info: Calling the main action
2020-08-06T07:10:36.325Z       stdout: 2020-08-06T07:10:36.325Z [main /adobeio/shared-validators-v1/headless] debug: params: {
   "source":{
      "mimetype":"application/octet-stream",
      "name":"20200514_085516.jpg",
      "size":1671305,
      "url":"https://XXXXXXXXX.blob.core.windows.net/firefly/source/20200514_085516.jpg?sv=2019-02-02&spr=https&se=2020-08-06T09%3A10%3A35Z&sr=b&sp=r&sig=XXXXXXXXXXX"
   },
   "__ow_method":"post",
   "userData":{
      "assetComputeClient":{
         "id":"74b0c0c0-d7b3-11ea-8b8c-df7dcfc540c3"
      }
   },
   "newRelicEventsURL":"https://insights-collector.newrelic.com/v1/accounts/2045220/events",
   "__ow_headers":{
      "accept":"*/*",
      "accept-encoding":"gzip, deflate",
      "authorization":"<hidden>",
      "connection":"close",
      "content-type":"application/json",
      "host":"controller-a",
      "perf-br-req-in":"1596697835.969",
      "user-agent":"node-fetch/1.0 (+https://github.com/bitinn/node-fetch)",
      "x-app-name":"Project_1596623723209J1",
      "x-forwarded-for":"X.X.X.X, X.X.X.X",
      "x-forwarded-host":"XXXXXXXXX-dev.adobeioruntime.net",
      "x-forwarded-port":"443",
      "x-forwarded-proto":"https",
      "x-gw-ims-org-id":"<MY_IMS_ORG>@AdobeOrg",
      "x-gw-ims-org-name":"<MY ORG NAME> Sandbox",
      "x-real-ip":"10.250.204.164",
      "x-request-id":"9zA6pY2h61iFaS9cmL8NGyd0m5Or2Ja7"
   },
   "times":{
      "core":"2020-08-06T07:10:35.849Z",
      "gateway":"2020-08-06T07:10:35.000Z",
      "process":"2020-08-06T07:10:35.817Z"
   },
   "__ow_path":"",
   "renditions":[
      {
         "name":"rendition.jpg",
         "target":"https://XXXXXX.blob.core.windows.net/firefly/rendition/20200514_085516.jpg/XXXXXXXXXX/0/rendition.jpg?sv=2019-02-02&spr=https&se=2020-08-06T09%3A10%3A35Z&sr=b&sp=cw&sig=XXXXXXXX&comp=block&blockid=XXXXXX",
         "userData":{
            "assetComputeClient":{
               "index":0,
               "length":1
            },
            "path":"rendition/20200514_085516.jpg/XXXXXXXX/0/rendition.jpg"
         },
         "worker":"https://XXXXXXXX-dev.adobeioruntime.net/api/v1/web/AEMMczAssetCompute-0.0.1/mcz-aem-asset-computer-worker?__VALIDATOR_LOG_LEVEL=debug"
      }
   ],
   "newRelicApiKey":"4iUx7ZvvK_PJ-FPLFDizxO3HllDa9LGN"
}
2020-08-06T07:10:38.901Z       stdout: 2020-08-06T07:10:38.901Z [main /adobeio/shared-validators-v1/headless] info: input IMS org id '<MY_IMS_ORG>@AdobeOrg' does not match app IMS org id 'true'
2020-08-06T07:10:38.902Z       stdout: 2020-08-06T07:10:38.902Z [main /adobeio/shared-validators-v1/headless] info: 403: request is invalid, failed authorization. Please use a valid JWT or user access token for this headless application.

[duypnguyen]

Hi Marcin - How did you call your service, from your local machine (CURL) or AEM? How did you get the access token? Please make sure that the required headers are available in the request: `Authorization` and `x-gw-ims-org-id`.

[sarahxxu]

In addition, for Asset Compute Workers, here is some documentation that may help. https://docs.adobe.com/content/help/en/asset-compute/using/extend/understand-extensibility.html