querying the profile-store

[Pradeep-Jaiswal]

is profile-store data exposed in any of the dataset ? is there any possibility to query the profile-store using query service ? except an API for a specific record, is there a way to get result of top 100 records from profile-store ?

[AnkitJasani29]

Hi @Pradeep-Jaiswal ,
Real‑Time Profile store isn’t exposed as a Query Service table. So, I can't find possibilities. Query Service only sees datasets in the Data Lake; the Profile store lives in its own service and must be accessed via the Profile APIs. One more thing which you can do is to first export profiles into a Data Lake dataset, then run your SELECT … LIMIT 100 against that new dataset in Query Service.

Reference:

https://experienceleague.adobe.com/en/docs/platform-learn/tutorials/queries/understanding-query-serv...

https://experienceleague.adobe.com/en/docs/experience-platform/profile/api/overview

https://experienceleague.adobe.com/en/docs/experience-platform/query/home

Thanks,

Ankit

[Pradeep-Jaiswal]

How to export profile stores into dataLake ? First we need to find all profile-Ids. Is there any api to get list of profile ids ? And then we capture the response of profile api into dataLake ?

[AnkitJasani29]

Hi @Pradeep-Jaiswal,

To dump every merged profile directly into a dataset in the data lake use profile export jobs API

https://experienceleague.adobe.com/en/docs/experience-platform/profile/api/export-jobs

For profile ids use below reference link

https://experienceleague.adobe.com/en/docs/experience-platform/profile/api/entities

Thanks,

Ankit

[Pradeep-Jaiswal]

once export-jobs is setup using API, I can see another set of API to check the Status. But can we also see profile-export status in the UI , would that reflect in the dataFlow tab or somewhere else ?

[AnkitJasani29]

Hi @Pradeep-Jaiswal ,

There isn’t yet an equivalent Profile exports panel in UI itself. All of the export‑job monitoring lives under Destinations > Data flows or System View. If you need per‑job metadata (run IDs, error details, etc.), continue to use the Export Jobs API - UI status lets you confirm success/failure at a glance, but the API will give you the full audit trail.

Thanks,

Ankit

[brekrut]

The profile store is a different storage layer than the datalake.  

Please refer to the following diagram.

[JoseePMM]

1. Is profile-store data exposed in any dataset?

No, profile-store data is not directly exposed in any dataset.
Adobe Experience Platform’s data architecture consists of several components:

• Data Lake: Raw ingested data, accessible via Query Service and datasets.
• Identity Graph: For identity resolution, stores relationships, not directly queryable through Query Service.
• Profile Store: Holds the stitched, unified profiles; this is the “profile-store.”

While datasets in the data lake are profile-enabled (meaning they feed data into the Profile Store), the stitched, unified, current state view of a profile as maintained by the Profile Service (profile-store) is not itself exposed as a dataset in the data lake.

“Platform’s key data stores are the Identity Graph and Profile Store, which are separate from the data lake. … You can’t use [Query Service] to query identity graphs or profiles since those are accessed using other tools like segmentation and the Identity Service.”
— Query Service and Data Distiller overview

2. Can you query the profile-store using Query Service?

No, you cannot query the profile-store directly with Query Service.
Query Service is limited to running SQL queries on data in the Data Lake—specifically, datasets. All interactive and scheduled queries are run on tabular data ingested into the Data Lake, not on the Profile Store’s real-time, stitched records.

• Query Service can:

  • Query behavioral and attribute datasets (such as web events, CRM data) in Data Lake.
  • Query profile-enabled datasets (raw records before ingestion into profile-store).

• Query Service cannot:

  • Query the real-time, merged/stiched profiles as maintained in Profile Store.
  • Access profile-store aggregates used by Real-Time Customer Profile (RCP) or segmentation.

Source:

• Query Service and Data Distiller overview

• 3. Can you get the top 100 records from profile-store, except via the profile API?

  No, there is currently no means to get “top N” queries from profile-store using SQL or Query Service.

  • The Profile API (entities endpoint) allows you to fetch a specific profile by identity (e.g., email, ECID).
  • There is no public API or UI mechanism to fetch a list of profiles or perform an unbounded scan (“top 100”) on the profile-store.
  • Profile UI and Segmentation UI allow you to browse profiles or sample “profiles in segment”, but this does not provide arbitrary “top N” access to all profiles.

  Workarounds:

  • For any “top N records” from a population, you must query the data lake (datasets), not the profile store.
  • To preview a dataset (not profile store), the Query Service UI now uses a SELECT * LIMIT 100 query against the dataset in the data lake (not profile store). Read more: Dataset preview in Experience Platform.

  See relevant documentation:

  • Real-Time Customer Profile Troubleshooting Guide

[Pradeep-Jaiswal]

is it fair to say the profileSnapshot export dataSet is a -1D copy of profile store ?

[arpan-garg]

Hi @Pradeep-Jaiswal  - You can't access the latest data in the profile store via queries but you can access the profile store data from the last day via querying the profile export snapshot which is created daily. It contains the profile store data from last day.