Do Processing Rules effect Data Warehouse? | Community
Skip to main content
Level 2
September 21, 2020
Solved

Do Processing Rules effect Data Warehouse?

  • September 21, 2020
  • 6 replies
  • 2004 views

Do Processing Rules effect Data Warehouse? I once had a processing rule which filtered out all numeric digits from all clicks.  We created this rule to avoid collecting account numbers (PII).  This processing rule worked as designed as no numeric digits appeared in UI data.  I was horrified to find account numbers in Data Warehouse as Data Warehouse ignored my processing rules.

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by isinha

Processing Rules are applied before Data Warehouse extracts. It's likely something was overlooked. Could you please check or run tests again?

6 replies

Jacob-DDdev
Level 6
September 21, 2020

Hi @erichami , how did you wind up filtering the numbers? Did you just create a "Does Not Contain" condition for every number 0-9? Just curious, but not sure about the actual answer. It is processed separately then the Analytics Reports/API so it seems plausible that it could have data that was filtered through normal processing.

You could try to filter before it reaches processing (e.g. custom code in your tag management -- even if the variable is set directly, you could copy it to a separate context data variable, use custom code to filter, then be sure to clear the analytics variable before sending the beacon, and, finally, set the processing rules to set the variable from the new, pre-filtered context data variable).

erichamiAuthor
Level 2
September 21, 2020
This was years ago, but the rule blocked any text that "contained" 0 or 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9. @jacob-dddev
Jacob-DDdev
Level 6
September 21, 2020
Thanks, @erichami . Yea, curious about this myself to know how normal processing impacts Data Warehouse.
Jacob-DDdev
Level 6
September 21, 2020

Now that I think about it, Data Warehouse must be available only after processing, since it includes values that are only set during processing (e.g. context data). It seems most likely that the conditional rule is failing. You may also want to do a Data Feed export on the variable(s) in question to compare the pre/post variable values to see how they were processed and if things are making it past the filtering processing rule.

isinhaAdobe EmployeeAccepted solution
Adobe Employee
September 22, 2020

Processing Rules are applied before Data Warehouse extracts. It's likely something was overlooked. Could you please check or run tests again?

jantzen_b
Adobe Employee
Adobe Employee
February 3, 2021
Do any of the answers below answer your initial question? If so, can you select one of them as the correct answer? If none of the answers already provided answer your question, can you provide additional information to better help the community solve your question?