Get data from 3rd endpoint and generate csv file

[neeliv]

Hi,

I am trying to fetch data from 3rd party endpoint and load it into one of data schemas in Adobe Campaign Standard.

The mentioned 3rd party endpoint has the option to put filters on query strings, so for instance it can looks like this:

var datenow = Date.now();
lasthour = now.setHours(now.getHours() - 1);

(date_modified >= lasthour) AND (date_modified < datenow)

To achieve this goal I've tried to use Adobe IO in the middle and I found that there is one similar use case described here: https://medium.com/adobetech/adobe-campaign-content-recommendations-in-email-b51ced771d7f

My script for Adobe IO looks as follows:

async function main () {
    try {
      //Declare all the options for GET request along with query parameters
      const fetch = require('node-fetch');
      const response = await fetch('https://endpoint.com', {
      	method: 'get',
        headers: {
          "Subscription-Key": "XXX",
          "TENANT": "YYY"
        }
      });
      const data = await response.json();

      console.log(data);

      //Generate current timestamps
      var datenow = Date.now();
      var lasthour = now.setHours(now.getHours() - 1);

      //Get the returned json and transform it into csv file
      if (data) {
        var json = data
        var fields = Object.keys(json[0])
        var replacer = function(key, value) { return value === null ? '' : value }
        var csv = json.map(function(row){
          return fields.map(function(fieldName){
            return JSON.stringify(row[fieldName], replacer)
          }).join(',')
        })
        csv.unshift(fields.join(',')) // add header column
         csv = csv.join('\r\n');

        return { message: csv }
      }

    } catch (error) {
    // log any server errors
    //logger.error(error);
    // return with 500
    return {
      statusCode: 500
    };
  }
}

Based on it I created the action and then api using wsk. However it doesn't work, it returns either 500 http code or 400 - depends which --response-type I'm using. I've tried to create api with all response types listed in documentation (html, http, json, text, svg).

I'd appriciate any help to make it running.

Thanks,

Dominik

[dhindle]

Hi Dominik,

What do the logs say? Does data contain a JSON value? Can you check on response.ok?

Maybe the endpoint is returning a message as its response, can you try

response.text()

if `response.json()` is empty.

Regards,

[chamir]

If the fields of data in your CSV file contain commas, you can protect them by enclosing those data fields in double quotes ("). Result in Sinuano Noche 2023, the commas that are part of your data are kept separate from the commas which delimit the fields themselves.