atrribute missing profilesnapshot dataset, exist in XDM union schema

Hi ​@SantoshRa5 ,

This is a known behavior in AEP related to how the ProfileSnapshot job populates the export dataset. A few things to check:

1. Schema merge policy and union schema propagation delay
Even though the attribute is visible in the union schema and Profile UI, there can be a delay between when a new field appears in the union schema and when the ProfileSnapshot job includes it in exports. The ProfileSnapshot job runs on a schedule and may not immediately reflect newly added attributes. Check when the field was added vs. when the last ProfileSnapshot job ran if the field was added after the last successful snapshot, trigger a new run and verify the next export.

2. Field-level inclusion in the export dataset schema
The ProfileSnapshot export dataset has its own schema that must include the field. Verify that the export dataset schema explicitly contains customer_number. Navigate to:
Datasets > ProfileSnapshot export dataset > Schema
Confirm customer_number appears there. If the export dataset schema was created before you added customer_number to your profile-enabled dataset, it may not have inherited the new field automatically.

3. Upsert vs. batch ingestion behavior
Since your dataset is upsert-enabled, verify the upsert records containing customer_number were fully processed and merged into the profile store before the snapshot ran. You can check this in Monitoring > Datasets > your dataset to confirm the upsert batches completed successfully with no partial failures.

4. Identity stitching scope
You mentioned customer_id exists in other datasets and helps with stitching. Confirm that the profile records being exported are the same identity-stitched profiles that contain customer_number if stitching is pulling in a different identity fragment that doesn't have customer_number, that field won't appear in the snapshot for those profiles.

Most likely this is either a snapshot timing issue (fix: wait for next run or trigger manually) or an export dataset schema not reflecting the new field (fix: update the export dataset schema to include customer_number).

​@SantoshRa5 

Profile snapshot dataset is not a real time dataset, it gets updated once a day most likely during your Sandbox’s Segment Evaluation time.

By design upsert works only for Profile and will be available for reference in Profile Lookup UI/API or Segments. Upsert does not update the data in datasets i.e. the data lake.

Hi ​@Devyendar  Thanks for your response.

The dataset is configured for upsert and profile is enabled. When I ingest records into this dataset, fields in it should appear in the profilesnapshot dataset after the daily batch evaluation runs.

However, even after successful ingestion in profilesnapshot, the records fieds do not appear in the profilesnapshot dataset and are therefore not included in the export.

Regards 

​@SantoshRa5 

Check if you see the field customer_number is showing up in ProfileSnapshot dataset even if it is blank. If yes that it is most likely you are seeing the wrong Profile Snapshot dataset. Each Merge policy creates a separate Profile Snapshot dataset you need to find the correct dataset for your merge policy

Use this query to get that info
SELECT
  merge_policy_name,
  merge_policy_id,
  dataset_id
FROM adwh_dim_merge_policies;

If this too doesn’t work. Refer to another Community post on similar issue they had to reach out to Adobe Support and Adobe confirmed that it was not Customer configuration issue but a snapshot creation issue and had to rebuild the snapshot on the backend.