How to manage changes on inbound data flow linked to existing schema that are currently in production with live data ingestion

[Luca_Lattarini]

Hello guys,

I have an existing schema in production with hourly ingestion data. Soon the inbound data will not provide an attribute which is currently mandatory. What is the best option to manage this?  Can I delete the field or make it optional?

Thanks

[vikash4 · Accepted solution]

Hi @Luca_Lattarini,

Since this is existing schema which might be associated with datasets having some data in it, making a field as optional could be a better option. Deleting a field from schema could make existing dataset incompatible.   

[ChetanyaJain]

@Luca_Lattarini 

If the schema is profile enabled then the best option is to make it non-mandatory as @vikash4 suggested. Here are some rules to understand the breaking changes https://experienceleague.adobe.com/docs/experience-platform/xdm/schema/composition.html?lang=en#evol...

To perform the cleanup, you will need to delete the entire set of records by deleting the dataset batches and then cleaning up the schema. Reconstruct the schema, re-create the API endpoint (if any) or reconfigure the batch ingestion/data mapping, etc and then ingest the data.

Thanks,

Chetanya