Marketo source connector can we put some filters on incoming data | Community
Skip to main content
S
Shaini_Gupta
Level 2
October 8, 2024
Solved

Marketo source connector can we put some filters on incoming data

  • October 8, 2024
  • 2 replies
  • 599 views

Hi All,

 

We are using Marketo source connector so just wanted to check if we can do some filtering or customization on the data which is getting ingested from Marketo connector.

 

Thanks & Regards

Shaini Gupta

This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
Best answer by Parvesh_Parmar

Hello @shaini_gupta ,

 

You can perform data transformation and validation before ingesting data into AEP.

 

1. Data Transformation: During the data mapping steps, when you set up your data flow, you can use built-in functions to transform the data. For example, if you have an email address in Marketo, you can encrypt in SHA256 it during ingestion.
For more details, please refer to the following links:

https://experienceleague.adobe.com/en/docs/experience-platform/data-prep/ui/mapping#add-a-new-field-type

 

https://experienceleague.adobe.com/en/docs/experience-platform/sources/ui-tutorials/create/adobe-applications/marketo#map-your-marketo-dataset-source-fields-to-target-xdm-fields

 

2. Data Validation: If you define any attribute as required in your schema, if there is a records missing that attribute during data ingestion, this record will be dropped. You should also set a threshold for error records. For example, if you set the threshold at 5% and more than 5% of records do not have the required attributes, the data flow will stop. You can enable partial ingestion to allow the process to continue with valid records.

https://experienceleague.adobe.com/en/docs/experience-platform/sources/ui-tutorials/create/adobe-applications/marketo#dataflow-configurations

 

 

Best regards,
Parvesh

2 replies

Parvesh_Parmar
Community Advisor
Parvesh_ParmarCommunity AdvisorAccepted solution
Community Advisor
October 8, 2024

Hello @shaini_gupta ,

 

You can perform data transformation and validation before ingesting data into AEP.

 

1. Data Transformation: During the data mapping steps, when you set up your data flow, you can use built-in functions to transform the data. For example, if you have an email address in Marketo, you can encrypt in SHA256 it during ingestion.
For more details, please refer to the following links:

https://experienceleague.adobe.com/en/docs/experience-platform/data-prep/ui/mapping#add-a-new-field-type

 

https://experienceleague.adobe.com/en/docs/experience-platform/sources/ui-tutorials/create/adobe-applications/marketo#map-your-marketo-dataset-source-fields-to-target-xdm-fields

 

2. Data Validation: If you define any attribute as required in your schema, if there is a records missing that attribute during data ingestion, this record will be dropped. You should also set a threshold for error records. For example, if you set the threshold at 5% and more than 5% of records do not have the required attributes, the data flow will stop. You can enable partial ingestion to allow the process to continue with valid records.

https://experienceleague.adobe.com/en/docs/experience-platform/sources/ui-tutorials/create/adobe-applications/marketo#dataflow-configurations

 

 

Best regards,
Parvesh

Parvesh Parmar – Adobe Community Advisor https://www.linkedin.com/in/parvesh-parmar/
    kautuk_sahni
    Community Manager
    kautuk_sahniCommunity Manager
    Community Manager
    October 16, 2024

    @shaini_gupta Did you find the suggestion helpful? Please let us know if you require more information. Otherwise, please mark the answer as correct for posterity. If you've discovered a solution yourself, we would appreciate it if you could share it with the community. Thank you!

    Kautuk Sahni
    Terms & ConditionsAccessibility statement

    Loading footer...