Need to know limitation on derived dataset created via queries.

[vvishnu0332]

Query 1: 

If the query output number is in the millions, will it be able to successfully ingest the data into a data lake, profile store, or identity services?

Test use case: An error was thrown when batch ingesting data larger than 20 GB.

Few limitation is available in the below link required some additional information.

Ref url: Export Datasets to Cloud Storage Destinations | Adobe Experience Platform

[jayakrishnaaparthasarathy · Accepted solution]

Hi @vvishnu0332  - Seems there is a limit up to 20 GB  of data per hour to data lake using the batch ingestion API.
Batch Ingestion Guardrails - https://experienceleague.adobe.com/docs/experience-platform/ingestion/guardrails.html?lang=en#:~:tex.... 
Default Guardrails - https://experienceleague.adobe.com/docs/experience-platform/profile/guardrails.html?lang=en 
Refer the above limits to avoid the error on your ingestion. 
Thank you,

Jay.

[hariprasathg]

for better ingestion of data through query service I think you need to have Data distiller SKU purchased.

https://experienceleague.adobe.com/docs/experience-platform/query/data-distiller/overview.html?lang=...

Thanks,

HariP