Expand my Community achievements bar.

Help shape the future of AI assistance by participating in this quick card sorting activity. Your input will help create a more effective system that better serves your needs and those of your colleagues.
SOLVED

How to enable a partial ingestion through API call?

Avatar

Level 2

Hi Team

We have a requirement to ingest the data from AWS S3 with encrypted data.so we are using API calls for decrypting the data through flow service API. Now our requirement is to enable partial ingestion with 1% threshold.

Through UI we were able to enable partial ingestion. We are looking for this configuration through API.

 

If someone help on this requirement will be much appreciated.

 

Regards,

Nikhil.N

1 Accepted Solution

Avatar

Correct answer by
Moderator

Hi @Nikhil70326 

 

When firing POST request for create flow can you structure the payload in this format and check -->

 

{
"name": "Flow",
"sourceConnectionIds": [
"XXXXXXXXXXXXX"
],
"targetConnectionIds": [
"XXXXXXXXXXX"
],
"flowSpec": {
"id": "XXXXXXXXXXXXX",
"version": "1.0"
},
"scheduleParams": {
"startTime": "1692086139",
"frequency": "once"
},
"options": {
"errorDiagnosticsEnabled": true,
"partialIngestionPercent": 20
},
"transformations": [
{
"name": "Mapping",
"params": {
"mappingId": "XXXXXXXXXXXXX",
"mappingVersion": 0
}
},
{
"name": "Encryption",
"params": {
"publicKeyId": "XXXXX"
}
}
]
}

 

Regards,

Arijit Ghosh

View solution in original post

1 Reply

Avatar

Correct answer by
Moderator

Hi @Nikhil70326 

 

When firing POST request for create flow can you structure the payload in this format and check -->

 

{
"name": "Flow",
"sourceConnectionIds": [
"XXXXXXXXXXXXX"
],
"targetConnectionIds": [
"XXXXXXXXXXX"
],
"flowSpec": {
"id": "XXXXXXXXXXXXX",
"version": "1.0"
},
"scheduleParams": {
"startTime": "1692086139",
"frequency": "once"
},
"options": {
"errorDiagnosticsEnabled": true,
"partialIngestionPercent": 20
},
"transformations": [
{
"name": "Mapping",
"params": {
"mappingId": "XXXXXXXXXXXXX",
"mappingVersion": 0
}
},
{
"name": "Encryption",
"params": {
"publicKeyId": "XXXXX"
}
}
]
}

 

Regards,

Arijit Ghosh