Mismatch between Query count and UI count.

[JettyRo]

I have created a new schema, and a new dataset (Event) corresponding to it , both are profile enabled .Created a new dataflow from the source and it is a backfill enabled.
Problem: I have ran a query to check the number of records in the dataset I got a count 'A'(2870051), but in the UI the count of records are 'B'(2,993,633).

PS: - there were some 4654 records failed during the first run of the dataflow, backfill enabled.
I am not understanding why this difference is.

[DavidRoss91]

Hi @JettyRo 

There potentially may be a delay between the ingestion and when the counts would match up. In the UI, you are checking in the monitoring section, correct? Also, about how long has it been since the data was ingested?

[JettyRo]

less than a week.

[nnakirikanti]

@JettyRo Is your source data deduped? Please check your query to remove distinct on _id field and also check the dataflow for more insights.

~cheers,

NN.

[JettyRo]

how can I check that (deduped), and I dont understand the logic of querying on _id, could you please eloberate.

[DavidRoss91]

@JettyRo @nnakirikanti  The _id value should be already unique per each event so i'm not sure that would be the case. Could you share your query logic?

[NickMannion]

What was the timeframe that you set for your audience and your query?

[kautuk_sahni]

@JettyRo Did you find the suggestions helpful? Please let us know if you need more information. If a response worked, kindly mark it as correct for posterity; alternatively, if you found a solution yourself, we’d appreciate it if you could share it with the community. Thank you!