Local File Upload for User Event Data in Adobe RTCDP

[yesraptors12]

Hi all,

I'm seeking guidance on uploading user event data via local file upload in Adobe RTCDP. Here's what I've done so far:

1. I've created an Event class schema along with a corresponding dataset.
2. I've successfully ingested this dataset using a local .csv file through the "local file upload" source.

However, the issue arises when I attempt to view the updated user profile. Despite the successful data ingestion, the user's latest event record doesn't appear to be reflected in their profile.

I've included some screenshots to provide more context. Could someone please advise on what might be going wrong?

Screenshots:
1) In event schema, I have selected personalEmail using namespace "Email"

2) Map data to XDM fields

 

3) Map data to XDM field as follow:

4) Data ingested to dataset successfully

5) Unable to find the user records under "Profiles"

Again, any suggestion on how to fix this is super welcome!

Thanks,

Rap

[Sneha-Parmar]

@yesraptors12 

Is the dataset profile enabled ?

[yesraptors12]

Hi @Sneha-Parmar ,

Yes, both Schema and Dataset is checked for Profile enabled.

Thanks,

Rap

[Neelabh_]

Hi Rap,

May not be a solution to your problem but a couple of pointers that may help you to figure out this issue.

01. For any non production sandbox, sometimes I have observed a delay in data being reflected in profile search UI when data is uploaded through offline file upload.

02. Try searching the record through other identities (if available in the record, like mobile number or record id) . If you can see the record, you can verify if email is present as an attribute to the same record. 

03. If above point works, try checking identity graph for same profile through other identities. Scroll through timeframe option on identity graph to see the identities as they we added to record through different ingestion time.

04.  Alternately point 02 & 03 can also be checked through query service. Try checking the record in dataset through any attribute. If found try checking for the same record in profile snapshot dataset to see the identities that are tied up with the particular record. A simple select query with table name (present in dataset) and attribute name = attribute value in where condition (where accordant_partner.test_specialrequirement=birthday) would suffice.

Hope this helps.

Thanks

Neelabh

[yesraptors12]

Hi @Neelabh_ & @Sneha-Parmar,

Unfortunately, I'm encountering difficulty locating the correct event records within the profile across different namespaces, whether I use email or mobile as identifiers.

I am currently working within a non-production sandbox environment. Could this be the source of the problem?

In below, I am trying to replicate the processes I did. If you notice any steps I may have missed, please let me know.

Here's what I've done to create the schema:

1. Created a new Schema, selecting "Experience Event".
2. Created a customized field group named "Manual Test" and added three new fields to it:
   • test_bookingDate (Type: Date)
   • test_numOfPersons (Type: Integer)
   • test_special (Type: String)
3. Within the same schema, selected the field group "Personal Contact Details", and under the "address" field under the personalEmail object, and checked both "Identity" and "Primary Identity". Also, In the Namespace dropdown, I chose "Email".
4. I saved and activated this schema by toggling the "Profile" slider to the on position.

Next, I created a Dataset:

1. Created a Dataset named "Manual test", applying the schema I created.
2. Saved this Dataset and also activated the "Profile" slider for it.

Lastly, I attempted to upload a file manually using the "local file upload" source:

1. uploaded the file below, please see the screencap:

     2. here's the mapping:

 
Please let me know if you can spot anything was wrongly implemented?

Thanks,

Rap

[brekrut]

Hello @yesraptors12 ,

I can see you have both records with the same datetimestamp but the id field is different.  This may have something to do with the issue, but I would recommend looking into the monitoring for the dataset which you have uploaded into Adobe RTCDP datalake. 

The screenshots provided appear to indicate data has been ingested into the datalake, but not promoted upon the unified profile store.  Can you provide screenshots of the monitoring tab for the time which the data was ingested into the platform. We should understand how many records have been selected for identity, how many ingested, and number of records for profile. 

[yesraptors12]

Hi @brekrut,

Thank you for guiding me in the right direction. In screenshot 3, I've noticed an error message. Could you advise on the steps I need to take to resolve it?

As far as I understand, I have already mapped the email to the "personalEmail" field using the "Email" namespace. Is there anything additional that I need to do regarding the "identityMap" field?

Error Code: UPINGT-007003-400

An error occurred while processing the batch. Please contact support.

RelatedEntityIdentityField: List(identityMap) is/are not present in the schema.

Screencap 1: The monitor screen for 2 manual file uploads:

Screencap 2: Clicked "Batch end-to-end" and find failure:

Screencap 3: Got the error message.

Thanks,

Rap

[brekrut]

You should not have to define the identitymap object with an identity field if you have mapped in email address as an identity field. 

The error message appears to indicate the schema was created initially without a primary identifier which could explain the identity map error message.  

I would try add in a identitymap field into your JSON test payload with the email address in the id map to see if this resolves the issue.

[yesraptors12]

Hi @brekrut,

Thanks a ton for your support!

I've noticed that the identityMap seems to pop up by default in my schema, and I can't quite figure out how to exclude it. Just to clarify, we haven't mapped anything to the identityMap object directly. Instead, we've been working with a different field named "address" that's part of the "personalEmail" object.

As for the JSON part, I'm a bit puzzled about how to proceed since our workflow requires us to use the "Local File Upload" source for .csv files.

Would anyone be willing to whip up a simple example .csv file for me? (Super simply one will do) I could use it to test and see what's different compared to my current file. I'm starting to think there might be a formatting issue with the file I'm uploading.

Thanks a bunch in advance for any help you can provide!

Here's the screencap for identityMap object, which we didn't mapped anything directly.

Here's the screencap for personalEmail object and we are using the field "Address" for identity.

By the way it look under the dataset review mode, the data is successfully ingested.

Thanks,

Rap

[brekrut · Accepted solution]

The identity map is an object which is setup by default when a time base schema is established.  The schema may have been setup and enabled for profile without a primary identity field defined.

It may make sense to create a new time series schema and only enable the email address field.