Your achievements

Level 1

0% to

Level 2

Tip /
Sign in

Sign in to Community

to gain points, level up, and earn exciting badges like the new
BedrockMission!

Learn more

View all

Sign in to view all badges

SOLVED

Do Processing Rules effect Data Warehouse?

erichami
Level 2
Level 2

Do Processing Rules effect Data Warehouse? I once had a processing rule which filtered out all numeric digits from all clicks.  We created this rule to avoid collecting account numbers (PII).  This processing rule worked as designed as no numeric digits appeared in UI data.  I was horrified to find account numbers in Data Warehouse as Data Warehouse ignored my processing rules.

data warehouse Processing Rules
1 Accepted Solution
ishans52004352
Correct answer by
Employee
Employee

Processing Rules are applied before Data Warehouse extracts. It's likely something was overlooked. Could you please check or run tests again?

View solution in original post

6 Replies
jkm-disco
Level 5
Level 5

Hi @erichami , how did you wind up filtering the numbers? Did you just create a "Does Not Contain" condition for every number 0-9? Just curious, but not sure about the actual answer. It is processed separately then the Analytics Reports/API so it seems plausible that it could have data that was filtered through normal processing.

You could try to filter before it reaches processing (e.g. custom code in your tag management -- even if the variable is set directly, you could copy it to a separate context data variable, use custom code to filter, then be sure to clear the analytics variable before sending the beacon, and, finally, set the processing rules to set the variable from the new, pre-filtered context data variable).

erichami
Level 2
Level 2
This was years ago, but the rule blocked any text that "contained" 0 or 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9. @jkm-disco
jkm-disco
Level 5
Level 5
Thanks, @erichami . Yea, curious about this myself to know how normal processing impacts Data Warehouse.
jkm-disco
Level 5
Level 5

Now that I think about it, Data Warehouse must be available only after processing, since it includes values that are only set during processing (e.g. context data). It seems most likely that the conditional rule is failing. You may also want to do a Data Feed export on the variable(s) in question to compare the pre/post variable values to see how they were processed and if things are making it past the filtering processing rule.

ishans52004352
Correct answer by
Employee
Employee

Processing Rules are applied before Data Warehouse extracts. It's likely something was overlooked. Could you please check or run tests again?

View solution in original post

jantzen_belliston-Adobe
Community Manager
Community Manager
Do any of the answers below answer your initial question? If so, can you select one of them as the correct answer? If none of the answers already provided answer your question, can you provide additional information to better help the community solve your question?