Today, we are going to talk about a method of onboarding data into RTCDP which is discussed a lot and helps stream data into Adobe Experience Platform from pretty much any third party. We are going to cover the HTTP API streaming source. So let’s get started.
Source
Navigate to the Sources in the Experience Platform UI.
![abhinavbalooni_0-1722313780373.png abhinavbalooni_0-1722313780373.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77710i50F41B230B1A2D3C/image-dimensions/793x543?v=v2)
Above, you would see ‘Set up’ the first time you try setting it up. Since, I’ve set up an account at my end, it shows up as Add Data.
Once you’ve created the account, you move on to Adding data. The screen looks something like below:
![abhinavbalooni_1-1722313780382.png abhinavbalooni_1-1722313780382.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77711i6EA45603CAF9F49A/image-dimensions/875x421?v=v2)
Go ahead and click on Add data.
![abhinavbalooni_2-1722313780387.png abhinavbalooni_2-1722313780387.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77712i58FF78F66DB8F746/image-dimensions/868x422?v=v2)
Select your dataset, you would like to stream data to. Review and finish.
![abhinavbalooni_3-1722313780392.png abhinavbalooni_3-1722313780392.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77713i3729E69917E675DC/image-dimensions/860x353?v=v2)
Next step is where most of the end users get confused. How to test whether the end point is working fine for them or not.
Testing Using Postman
Next step is to get the streaming endpoint details to stream test data to. Select your dataflow from the list of dataflows for the account. Scroll to the bottom in the details that show up on the right of the screen. You should see details for using with the API endpoint. Something like the below:
![abhinavbalooni_4-1722313780393.png abhinavbalooni_4-1722313780393.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77714i08C2E97E1FF20569/image-dimensions/337x462?v=v2)
Open up postman, and create a new POST request. Feed the streaming endpoint from above into the request url. The other two values, dataflow id and the dataset id will come in handy when creating the POST Body request.
In my example, I am trying to update a consent field group in my profile. You can do pretty much anything you want to with the streaming endpoint.
My payload looks something like the below (yours would look different depending on your schema)
{
"header": {
"schemaRef": {
"id": "https://ns.adobe.com/spnam/schemas/<put in your schema id here> ",
"contentType": "application/vnd.adobe.xed-full+json;version=1"
},
"flowId": "<feed the dataflow id from the dataflow details here",
"datasetId": "<feed the dataset id from the dataflow details here>"
},
"body": {
"xdmMeta": {
"schemaRef": {
"id": "https://ns.adobe.com/spnam/schemas/<put in your schema id here> ",
"contentType": "application/vnd.adobe.xed-full+json;version=1"
}
},
"xdmEntity": {
"_id": "1112121",
"_repo": {
"createDate": "2024-07-28T12:00:00-06:00",
"modifyDate": "2024-07-28T12:00:00-06:00"
},
"consents": {
"collect": {
"val": "y"
},
"idSpecific": {
"ECID": {
"<ECID value removed from here>": {
"adID": {
"idType": "IDFA",
"val": "y"
},
"collect": {
"val": "y"
},
"marketing": {
"email": {
"reason": "Sample value",
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"push": {
"reason": "Sample value",
"time": "2024-05-15T20:20:39+00:00",
"val": "n"
},
"sms": {
"reason": "Sample value",
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
}
},
"personalize": {
"content": {
"val": "y"
}
},
"share": {
"val": "y"
}
}
}
},
"marketing": {
"any": {
"reason": "Sample value",
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"call": {
"reason": "Sample value",
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"commercialEmail": {
"reason": "Sample value",
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"email": {
"reason": "Sample value",
"subscriptions": {
"key": {
"subscribers": {
"key": {
"source": "Sample value",
"time": "2018-11-12T20:20:39+00:00"
}
},
"topics": [
"Sample value"
],
"type": "Sample value",
"val": "y"
}
},
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"fax": {
"reason": "Sample value",
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"postalMail": {
"reason": "Sample value",
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"preferred": "email",
"push": {
"reason": "Sample value",
"subscriptions": {
"key": {
"subscribers": {
"key": {
"source": "Sample value",
"time": "2018-11-12T20:20:39+00:00"
}
},
"topics": [
"Sample value"
],
"type": "Sample value",
"val": "y"
}
},
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
},
"sms": {
"reason": "Sample value",
"subscriptions": {
"key": {
"subscribers": {
"key": {
"source": "Sample value",
"time": "2018-11-12T20:20:39+00:00"
}
},
"topics": [
"Sample value"
],
"type": "Sample value",
"val": "y"
}
},
"time": "2018-11-12T20:20:39+00:00",
"val": "y"
}
},
"metadata": {
"time": "2018-11-12T20:20:39+00:00"
},
"personalize": {
"content": {
"val": "y"
}
},
"share": {
"val": "y"
}
},
"createdByBatchID": "/uri-reference",
"identityMap": {
"ECID": [
{
"authenticatedState": "ambiguous",
"id": "<ECID value removed from here>",
"primary": true
}
]
}
}
}
}
Once you have the POST body request ready. Your request should look something like the below:
![abhinavbalooni_5-1722313962088.png abhinavbalooni_5-1722313962088.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77718i34B53F61A1CEF961/image-dimensions/860x503?v=v2)
Now go ahead and press send. If your request is correctly structured, you will see a 200 OK response like below.
![abhinavbalooni_6-1722313962100.png abhinavbalooni_6-1722313962100.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77717i9BBAE06F38AC601E/image-dimensions/836x422?v=v2)
Once done, it should show up in the dataflow runs in a few minutes. You can check the profile and should see the latest data reflecting near real time. Have a look at the highlighted attributes which show the latest hit reflecting in my case.
![abhinavbalooni_7-1722313962106.png abhinavbalooni_7-1722313962106.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77716iE1A0C07FA0D32C13/image-dimensions/866x241?v=v2)
You can also verify that the ingestion was successful by looking at the dataflow run and should also show up in the dataset as a batch ingestion. As you can see below, the dataflow is processing the hit that was sent through.
![abhinavbalooni_8-1722314026131.png abhinavbalooni_8-1722314026131.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77720iE678D650EE22DD92/image-dimensions/883x272?v=v2)
Dataset also should show up the batch ingestion in the overview like the one below:
![abhinavbalooni_9-1722314129007.png abhinavbalooni_9-1722314129007.png](https://experienceleaguecommunities.adobe.com/t5/image/serverpage/image-id/77722i4B95CC939041CFA5/image-dimensions/808x457?v=v2)
That is pretty much what is there to it ! Happy Streaming data into AEP !
If you would like to have a look at the earlier posts from this series, please find the links below:
Week 3: https://experienceleaguecommunities.adobe.com/t5/real-time-customer-data-platform/tuesday-tech-bytes...
Week 2: https://experienceleaguecommunities.adobe.com/t5/real-time-customer-data-platform/tuesday-tech-bytes...
Week 1: https://experienceleaguecommunities.adobe.com/t5/real-time-customer-data-platform/tuesday-tech-bytes...
Come back for more next week.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.