Incredibly slow Identity and profile resolution

[marc_stowe_za]

I'm wondering if there is some form of platform setting that throttles the rate of data being processed for Identity and / or profile resolution.

I followed this process.

1. created data schemas for a primary profile CRM load and 2 event type data sets.

2. created a connector to Dynamics and loaded all the data across into data sets.

3. checked the data sets to be sure I was happy with the content

4. ensured the same primary identity in all 3 data sets (its CRM Id Guid), then switched both the schemas and the data sets to 'Profile'

5. CDP starts building the profiles and identity stitching

Problem is, its only processing about 35k records every morning at 1am, then nothing for the rest of the day. At this rate it will take a month to process 1 million records. Surely I need to configure something that tells the data lake to get cracking, or could this be a default development sandbox type of issue?

thanks in advance for any advice

[marc_stowe_za]

quick update on this. Still no idea why the CDP engine was taking its time, and dont have a month to wait, so removed the datasets and reloaded them. All 950k odd profiles loaded, reloaded the events and all are present, identities are stitched

My thinking is this. If the schema and datasets are activated to profile after they are loaded, the processing engine seems to process slowly in the background (this still needs a decent explanation), but when the profile has already been activated on schema and dataset, it builds immediately (ish)

This doesnt answer my problem, as I'm sure the best way to incorporate data would be to do like I did in the question above (ie bring the data in, make sure you are happy with the result and then switch it into profile management)

[brekrut]

Hello

Is this being done against a PROD or a non-PROD sandbox.  Typically a non-PROD sandbox does not have the same resources as PROD sandbox.