Failed records(batch ingestion) due to MaxIdentities and InvalidXdm error

[SahuSa1]

Hi,

There are few records which have failed to ingest (to a dataset having batch ingestion) due to MaxIdentities error and InvalidXdm error. For MaxIdentities it says "Identities size is more than defined limit 20 for Ingestion" and for InvalidXdm it says "Invalid ECID found. Skipping this record".

Could someone please help me understand what does the MaxIdentities error mean? And if there is any way to stop these records from failing?

  

Thanks,

Sambit

[KumarRishii]

Both MaxIdentities and InvalidXdm errors are common in batch ingestion pipelines in AEP.

To fix MaxIdentities Error:

Limit identity types to 20 or fewer per record.
Remove unused or duplicate identity namespaces.
Add pre-ingestion validation to filter or truncate excess identities.

To fix InvalidXdm (Invalid ECID) Error:

Ensure ECID is a valid, non-empty string (usually 38 characters).
Confirm the identity namespace is exactly "ECID" (case-sensitive).
Validate and clean ECIDs before ingestion.

I will suggest you to  add pre-validation logic to your ETL or Data Prep pipeline to catch and correct these issues before ingestion.

[SahuSa1]

Hi @KumarRishii , Thanks for your quick reply.

I have following doubts-

Q1. The schema on which the dataset is built has 3 identities in total.. so identity type which is limited to 20, does that mean the maximum no of different identity namespaces coming from sources should be 20 max or is it the the number of identities of same identity namespace should be 20 max? 

For example-

Case 1: 20+ different identities from 20+ identity namespaces.

Case 2: 20+ records from one or two identity namespace.

Which one would be the correct interpretation of MaxIdentities error? 

Q2. Above methods suggested by you is only possible before data lands in AEP, right? 

Thanks,

Sambit

[KumarRishii]

Identity type limited to 20" means you can have at most 20 distinct identity namespaces/types. You can have many records/values per type, but only 20 different types/namespaces.

Yes, the methods I suggested (identity cleanup, ECID validation, etc.) are all pre-ingestion solutions — meaning they need to be implemented before the data lands in AEP.

I hope the above reference helps clarify your queries.

[SahuSa1]

Hi @KumarRishii , Thanks for your reply.

I checked with Adobe support and also checked the failed batch's contents using API and found that there were 20+ values from the same identity namespace due to which those records were skipped.

So, to sum it up there cannot be more than 20 identities in a single XDM record be it from a single identity namespace or multiple identity namespaces. 

Thanks, 

Sambit