Streaming test via HTTP API has stopped ingesting the data in Profile Store | Community
Skip to main content
M
mustufam5967803
Level 3
February 20, 2026
Solved

Streaming test via HTTP API has stopped ingesting the data in Profile Store

  • February 20, 2026
  • 2 replies
  • 65 views

Hello Team,

When I am ingesting the data via HTTP API Source the data is getting ingested in dataset but not in profile store, however it was working fine earlier. 

Current Observations:

When I check dataflow runs it show completed
 

 

But in the dataset I see two entries getting created for same record one with failed and other with success. The success records do not get ingested in Profile Store
 

 

Troubleshooting: I did following troubleshoot however no luck:

  1. Created a new HTTP API data flow as the earlier one was not working, the new worked well for the first time and next day it has stopped working
  2. I made sure that the dataset is profile enabled
  3. I validated the success dataset batch records via query services and they are appearing fine in the dataset however not reflecting in profile store
  4. I made sure the payload has all the mandatory fields such as primary and secondary keys

Could anyone confirm how to fix this, this would be a help.

Thanks in advance!

 

Regards,

Mustufa

 

 

Best answer by arpan-garg

Hi ​@mustufam5967803 - Can you click on the failed batch and check the error there? Does your identityMap has two primary identities?

2 replies

SG14_97
SG14_97
Level 3
February 20, 2026

Hi ​@mustufam5967803 ,

Streaming works a bit differently as compared to batched ingestions method where the primary datastore is datalake and then internal jobs perform the identity resolution and subsequently promotes it to UPS. When streaming your data should reflect in near real time in UPS first and datalake replication might be delayed because it mircobatches multiple incoming events into one batch and then you see the processing at the datalake level.

That being said when you say the data is not in profile store do you mean you cannot find that identity altogether in the UPS or you are inferring it from the datalake stats?

M
mustufam5967803Author
Level 3
February 20, 2026

Thanks for the reply ​@SG14_97, 

Just to be clear this is not based on data lake replication timing or micro batch delays. The records are successfully landing in the data lake dataset but the identities are not appearing in the UPS. This has been happening consistently for the past two days and the same streaming setup was working earlier.

I made sure that Identitymap is properly passed with all the fields. I have also tested this using other colleague payloads but the result is still the same. 

SG14_97
SG14_97
Level 3
February 21, 2026

Thanks ​@mustufam5967803 as a double check can you validate the ingestion timestamp associated with the event and if any TTL is applied for this event ingestion. Might be affected by the TTL.

Also use the batch endpoint to get some details from the API on the status of the batch or alternatively check the flow run in case something is going wrong.

arpan-garg
Community Advisor
arpan-gargCommunity AdvisorAccepted solution
Community Advisor
February 20, 2026

Hi ​@mustufam5967803 - Can you click on the failed batch and check the error there? Does your identityMap has two primary identities?

M
mustufam5967803Author
Level 3
February 23, 2026

Hello Arpan, 

Thank you for the reply, I checked the failed batch error and it says “Identity not available”, following is the response for reference.

 "errors": [
            {
                "code": "UPINGT-030077-400",
                "description": "Identity not available.",
                "rows": []
            }
        ]

 

However, my payload has identifymap fieldgroup details added which has same id details as my primary identity details, following is the sampe for reference:

 

  "identityMap": {

                "key": [

                    {

                        "authenticatedState": "authenticated",

                        "id": "xxx@xxx.com",

                        "primary": true

                    }

                ]

            },

 

"workEmail": {

                "address": "xxx@xxx.com",

                 "primary": true

}

 

 

 

 

itsMeTechy
itsMeTechy
Level 4
February 23, 2026

when you say for the first time ingestion it works fine. can you please post the identityMap values for that ingestion. 

what is the namespace you are using?

in your schema, is identityMap is the only identity? or you have also marked any other attributes as identity? if so is there any validation in the attribute like pattern of data should be proper Email like that. 

what is the _id you are using in the consecutive ingestion. if the _id is same the record will be ingested into datalake but not UPS.

Terms & ConditionsAccessibility statement

Loading footer...