Data Modeling and Event Limitations

[DavidRoss91]

Hi All

I have a few questions I wanted to pose:

1. Should there be a limitation on event types from the source data that will exist in our specific schema (AEP Event Schema) in this example?  Any concerns if this schema continues to grow without limitation on objects existing in Event Classification as seen below? I.e. we continue to add more objects as needed under Event Classification.

2. If the schema grows with each use case/source event (ex: Customer Awareness, Connection Test, Change Stages, etc.) should we anticipate any performance issue if the classification object extends, but is comprised of mostly blank or unused values? 

3. Are you aware of any limitations in AEP for total events that can be ingested daily/weekly?

@brekrut 

[ccg1706 · Accepted solution]

Hi @DavidRoss91 ,

Happy to answer the questions with my experience and the knowledge that I have until the moment.

1. Regarding limitations on Event Types in AEP Event Schema it is generally recomendable to impose limitations on the types of events from the source data that will exist in your specific AEP Event Schema. Unrestricted growth may lead to schema complexity, making it harder to manage and utilize it effectively. It can also derive on performace and scalability , as a bigger schema may include many unsued or irrelevant fields leading to inefficient data storage and retrieval processes. It's crucial to evaluate and consolidate events, being sure that each added event type is necessary and aligns with your business goals and analytics requirements.

2. Performance Issues with Growing Schema. If your schema grows with each new use case or source event, extending the classification object but comprising mostly blank or unused values, you might encounter performance issues. These inefficiencies might manifest in slower query performance, icreaded storage costs , and more challenging data management. To mitigate this, regularly view and optimize your schema, ensuring it remains streamlined and relevant.

3. Until what I know, AEP has certain limitations. The platform does enforce various guardrails and performance recommendations to ensure optimal operation. It is crucial to plan and monitor your data ingestion to stay within these operational thresholds.

¡Hope this is helpful!

@jpetermarias 

Regards,

Celia