Expand my Community achievements bar.

SOLVED

How to manage changes on inbound data flow linked to existing schema that are currently in production with live data ingestion

Avatar

Community Advisor

Hello guys,

 

I have an existing schema in production with hourly ingestion data. Soon the inbound data will not provide an attribute which is currently mandatory. What is the best option to manage this?  Can I delete the field or make it optional?

Thanks

1 Accepted Solution

Avatar

Correct answer by
Level 3

Hi @Luca_Lattarini,

Since this is existing schema which might be associated with datasets having some data in it, making a field as optional could be a better option. Deleting a field from schema could make existing dataset incompatible.   

View solution in original post

2 Replies

Avatar

Correct answer by
Level 3

Hi @Luca_Lattarini,

Since this is existing schema which might be associated with datasets having some data in it, making a field as optional could be a better option. Deleting a field from schema could make existing dataset incompatible.   

Avatar

Community Advisor

@Luca_Lattarini 

If the schema is profile enabled then the best option is to make it non-mandatory as @vikash4 suggested. Here are some rules to understand the breaking changes https://experienceleague.adobe.com/docs/experience-platform/xdm/schema/composition.html?lang=en#evol...

 

To perform the cleanup, you will need to delete the entire set of records by deleting the dataset batches and then cleaning up the schema. Reconstruct the schema, re-create the API endpoint (if any) or reconfigure the batch ingestion/data mapping, etc and then ingest the data.

 

Thanks,

Chetanya