Introduction
This blog post explores the data egress cases offered by Adobe Experience Platform (AEP) and Customer Journey Analytics (CJA). We'll examine each option's diverse functionalities, providing you with a clear understanding of their capabilities. We'll also analyze the strengths and potential limitations of each method, empowering you to make informed decisions when selecting the most suitable egress option for your specific needs and requirements.
Our comprehensive review examines a broad spectrum of factors, including data processing capabilities, integration potential with external systems, scalability for handling large datasets, and user-friendliness of extract initiation. By thoroughly exploring these aspects, we aim to provide you with a robust framework for evaluating and comparing the diverse egress options available within AEP and CJA.
To provide an up-to-date comparison of the various AEP and CJA data egress options as of October 2024, here's a summary matrix highlighting key aspects of each method:
| AEP/CJA Data Egress Option | Extract Limits | Extract Initiated From | Key Advantages | Main Limitations |
|---|---|---|---|---|
| 1. AEP: Export Datasets | Varies based on destination cloud storage limits | AEP UI or Flow Service API | Raw data access, flexible scheduling, JSON or Parquet formats | Licensed functionality for available datasets, requires supported cloud storage destination setup or SFTP connection |
| 2. AEP: Query Service | 50,000 rows per query (standard with no override), 10-minute query timeout | AEP Query Service UI, connected client through PostgreSQL protocol, or API | SQL-based exploration, flexibility in query creation, use of familiar SQL tools | Requires SQL knowledge, potential resource constraints limits. |
| 3. CJA: SQL BI Extension | 50,000 rows per query, 6-minute query timeout | Direct SQL or External BI tools (e.g., Tableau, Power BI) | PostgreSQL integration with BI tools, combine CJA data with external sources, customized report creation | Requires SQL/BI tool knowledge, requires date range limit and aggregate queries, potential resource constraints limits. |
| 4. CJA: Workspace UI | 50,000 rows (CSV via Download items as CSV), 400 rows (Download as CSV or PDF) | CJA Analysis Workspace UI Download | User-friendly interface, intuitive data exploration, wide range of visualization options | Limited egress options, not for large-scale egress |
| 5. CJA: Reporting API | 50,000 rows per API call | Reporting API calls | Flexibility in data retrieval, automation of reporting, scalable extraction, integration with custom applications | Requires programming skills, potential breakdown and rate limits |
| 6. CJA: ReportBuilder | 50,000 rows per Excel data block request | Excel add-in | Familiar Excel interface, flexible report customization, easy sharing and scheduling | Excel format-based only, limited dataset size |
| 7. CJA: Full Table Export | 3 million, 30 million, 150 million, or 300 million rows per egress (based on CJA license tier) | CJA Analysis Workspace UI Export full table | Access to fully processed CJA data, includes advanced features like attribution, flexible egress scheduling | CJA-processed data with a maximum of 5 dimensions and 5 metrics. This can result in large CSV or JSON files delivered daily to a cloud export account. |
This matrix provides a quick overview of each egress option, highlighting the extract limits, where the extract is initiated, and key advantages and limitations. When choosing an egress method, consider your specific data needs, technical expertise, and integration requirements to select the most appropriate option for your use case.
To further assist in choosing the most suitable egress option for your specific needs, here's a matrix summarizing the primary use cases for each of the seven egress options.
| AEP/CJA Data Egress Option | Data Lifecycle Stage | Primary Use Cases |
|---|---|---|
| 1. AEP: Export Datasets | AEP raw unprocessed data | - Exporting raw, unprocessed CJA data from AEP - Integrating AEP data with cloud storage solutions - Long-term data archiving and compliance |
| 2. AEP: Query Service | AEP raw unprocessed data | - Complex SQL-based data exploration and analysis - Data validation and quality checks - Creating custom datasets for specific business needs - Integrating with SQL-compatible tools and platforms |
| 3. CJA: SQL BI Extension | CJA processed data | - Integrating CJA data via PostgreSQL or external BI tools (e.g., Tableau, Power BI) - Creating comprehensive dashboards combining CJA and other data sources - Advanced visualizations and reporting in preferred BI platforms |
| 4. CJA: Workspace UI | CJA processed data | - Ad-hoc data exploration and visualization - Creating interactive dashboards - Quick insights and presentations for stakeholders |
| 5. CJA: Reporting API | CJA processed data | - Automated report generation and data extraction - Integration with custom applications and dashboards - Programmatic access to CJA data for advanced analytics - Scheduled data egress for regular reporting needs |
| 6. CJA: ReportBuilder | CJA processed data | - Creating customized Excel reports with CJA data - Scheduling automated Excel-based report distribution - Combining CJA data with other Excel-based analyses |
| 7. CJA: Full Table Export | CJA processed data | - Exporting large volumes of processed CJA data - Accessing persisting dimensions, attribution, and sessionized data - Performing deep-dive analyses in external tools |
This matrix provides a quick reference to help you identify which egress option aligns best with your specific use cases and requirements. Consider factors such as data processing needs, integration capabilities, and the level of customization required when selecting the most appropriate egress method for your organization.
Customer Journey Analytics' Report-Time Processing: Impact
Report-time processing, profoundly shapes data egress. This dynamic approach impacts both data analysis and the interpretation of egressed data:
- Flexible Data Collection:
- Events aren't necessarily collected in chronological order
- Individuals are identified by a declarative person ID, enabling accurate user tracking across devices and channels
- Dynamic Attribution and Sessionization:
- CJA accounts for persistence, attribution, and sessionization at report time, not at data collection time
- This allows for more flexible and accurate analysis as new data is incorporated
- Impact on Data Egress:
- Egressed data may vary depending on the egress time, as new data added to the CJA Connection can affect historical aggregation, persisted values, and attribution
- Stitching replay capabilities can link anonymous events with known authenticated events, potentially altering historical data interpretations.
- Egress from different times may show variations in metrics due to the dynamic nature of report-time processing
Understanding these impacts is crucial when interpreting and utilizing data egress from CJA. The dynamic nature of report-time processing can lead to evolving insights over time, offering a more nuanced and up-to-date view of your latest data landscape.
1. AEP: Export Datasets
Summary: This functionality allows users to egress data collected within AEP directly to various supported cloud storage or SFTP destinations. By using this feature, organizations can efficiently transfer large volumes of raw data from AEP to their preferred cloud environments, enabling further analysis, integration with other systems, or long-term data archiving. This direct export capability streamlines data management processes and enhances data accessibility and utilization across different cloud-based tools and services. Ultimately, it empowers businesses to derive deeper insights and make more informed decisions based on their comprehensive AEP datasets.
Data Granularity: Row-Based
Dimension Breakdown Limit: Not Applicable
Egress Destinations: Cloud Storage, SFTP
Licensing/SKU Note: Available in All CJA Entitlement Packages
Access: Granted by the 'Destinations: Manage and Activate Dataset Destinations' AEP Resource permission
Functionalities:
- Schedule or ad-hoc exports
- Egress to various cloud destinations
- Access to raw, unprocessed data that serves as the foundation for CJA
Benefits:
- Direct access to AEP underlying datasets
- Flexible scheduling options
- Integration with supported cloud storage solutions
Considerations:
- Requires cloud storage setup
- Datasets not yet processed through CJA
- Datasets available for export depend on your Experience Platform application (such as Real-Time CDP or Adobe Journey Optimizer), your subscription tier (Prime or Ultimate), and any additional add-ons you've purchased (like Data Distiller).
2. AEP: Query Service
Summary: This feature enables users to execute complex SQL queries on their AEP datasets within the platform. By leveraging this functionality, users can perform sophisticated data analysis and exploration directly on their stored AEP data. The Query Service empowers both data analysts and developers to harness the full potential of their datasets through custom SQL queries, facilitating deeper insights and more granular control over data extraction and transformation processes.
Data Granularity: Row-Based
Dimension Breakdown Limit: Not Applicable
Egress Destinations: Not Applicable
Licensing/SKU Note: CJA Select package or higher required
Access: Granted by the ‘Query Service: Manage Queries’ AEP Resource permission
Functionalities:
- Ad-hoc SQL query execution
- Data validation
- Integration with SQL tools
Benefits:
- SQL-based data exploration along with pre-built Adobe-defined functions
- Flexibility in query creation
- Ability to use familiar SQL tools
Considerations:
- Requires SQL knowledge
- May be complex for non-technical users
- Potential for resource-intensive queries, especially for those new to SQL
- Post-ingestion data preparation through Batch Queries and Ad Hoc Queries requires an additional license for the Data Distiller Add-on
- Python module available for Query Service API via the aepp AEP wrapper
3. CJA: SQL BI Extension
Summary: This feature enables PostgreSQL integration between CJA Data Views and popular external BI/visualization tools, such as Power BI and Tableau. Users can incorporate CJA-processed dimensions and metrics into their preferred business intelligence platforms, enhancing their ability to create comprehensive, data-driven visualizations. This integration blends CJA's advanced analytics capabilities with the robust reporting and dashboard creation features of external BI tools. As a result, users can generate insightful and visually compelling reports that combine CJA data with other business metrics and key performance indicators.
Data Granularity: Aggregated
Dimension Breakdown Limit: 5 Breakdowns
Egress Destinations: BI Tools
Licensing/SKU Note: CJA Select package or higher required
Access: Granted by the 'Data View Tools: CJA BI Extension' CJA Product Profile permission
Functionalities:
- PostgreSQL-connection integration via direct SQL and BI tools
- Combine CJA data with external data sources
- Customized report creation with specified date ranges and aggregated data queries
Benefits:
- Utilize familiar BI tools like Power BI or Tableau
- Combine CJA data with other business data
- Create comprehensive BI/visualization dashboards
Considerations
- Requires knowledge of external BI/visualization tools
- May require additional licenses for BI/visualization tools outside of Adobe's ecosystem
- Constrained by the specific features and functionalities of the selected BI/visualization tool
4. CJA: Workspace UI
Summary: The CJA Analysis Workspace UI offers a dynamic, drag-and-drop interface that empowers users to craft, refine, and customize reports and visualizations at the Data View level. This user-friendly platform facilitates data exploration and analysis, enabling both novices and experts to transform complex datasets into actionable insights. Users can easily manipulate dimensions, metrics, and filters to build tailored reports and project exports that address specific business questions. The Workspace UI's flexibility fosters rapid iteration and experimentation, allowing swift adaptation of analyses as new insights emerge or business needs evolve. It supports direct download export and scheduled export for enhanced data sharing and reporting capabilities.
Data Granularity: Aggregated
Dimension Breakdown Limit: No Defined Limit
Egress Destinations: Direct UI Download, Email
Licensing/SKU Note: Available in All CJA Entitlement Packages
Access: Granted by the 'Reporting Tools: Analysis Workspace Access' CJA Product Profile permission
Functionalities:
- Interactive data visualization
- Ad-hoc analysis capabilities
- Segmentation and calculated metrics creation
- Support for various data formats (e.g., PDF, CSV)
- Share a project with anyone (no login required) to provide read-only access to Analysis Workspace projects for individuals without CJA access
Benefits:
- User-friendly interface simplifies the creation of basic exports
- Intuitive data exploration through interactive panels with scheduling options
- Wide range of visualization options
Considerations:
- Limited egress options compared to other methods
- May require training and CJA implementation understandings
- Not suited for automated, large-scale data exports
5. CJA: Reporting API
Summary: The CJA Reporting API is a tool that allows developers and data analysts to programmatically access and retrieve data from CJA. This API enables the creation of custom applications, integrations, and automated reporting solutions by providing a standardized method to query and extract CJA data. By leveraging this API, organizations can incorporate CJA insights into their existing systems, dashboards, or custom-built applications, facilitating more efficient data-informed decision-making processes across the organization.
Data Granularity: Aggregated
Dimension Breakdown Limit: 1 Dimension Level Per Request (Breakdowns are treated as dimensions, with filters applied to each metric)
Egress Destinations: Custom Applications, External Systems, Data Warehouses, Dashboards
Licensing/SKU Note: Available in All CJA Entitlement Packages
Access: Granted by Adobe Admin Console Developer permission, API Client, and OAuth Authentication
Functionalities:
- Programmatic access to CJA data
- Custom report generation
- Integration with external systems and applications
Benefits:
- Flexibility in data retrieval and manipulation
- Automation of reporting processes
- Scalable data extraction through pagination
- Integration with custom applications and workflows
Considerations:
- Requires programming skills and API knowledge
- Usage rate limits, breakdown limits, or usage restrictions
- Potential challenges in managing large datasets or intricate nested reports
- cjar R Client and cjapy Python wrapper for easier API integration
6. CJA: ReportBuilder
Summary: CJA ReportBuilder is a user-friendly tool that empowers users to create customized, data reports by leveraging CJA data within the environment of Microsoft Excel. This integration is facilitated through a specialized Excel add-in, which serves as a bridge between the analytics capabilities of CJA and the versatile spreadsheet functionalities of Excel. By utilizing CJA ReportBuilder, users can access, analyze, and visualize complex customer journey data, transforming information into actionable insights and compelling Excel-based reports tailored to their specific business needs.
Data Granularity: Aggregated
Dimension Breakdown Limit: 2 Breakdowns
Egress Destinations: Email
Licensing/SKU Note: Available in All CJA Entitlement Packages
Access: Install CJA Report Builder Excel Add-in
Functionalities:
- Create custom reports in Excel
- Schedule and automate email report distribution
- Access to CJA dimensions and metrics
Benefits:
- Familiar Excel interface for many users
- Flexible report Excel-based customization
- Easy sharing and scheduling of reports via email
Considerations:
- Limited to Excel format
- May require some Excel expertise for advanced features
- Not suitable for very large datasets
7. CJA: Full Table Export
Summary: This advanced CJA export feature allows users to export large, processed datasets from CJA, including persisted dimensions, calculated metrics, attribution, and sessionization. Organizations can access refined data that has been processed and unified within CJA, providing valuable insights into customer behaviors, interactions, and journeys across various platforms and touchpoints. This processed data enables organizations to make more informed decisions and optimize their strategies.
Data Granularity: Aggregated
Dimension Breakdown Limit: 5 Breakdowns
Egress Destinations: Cloud Export Account (AEP Data Landing Zone, Cloud Storage [Amazon S3 Role ARN, Google Cloud Platform, Azure SAS, Azure RBAC], Snowflake)
Licensing/SKU Note: 3 million, 30 million, 150 million, or 300 million rows per export (based on CJA Entitlement Package)
Access: Granted by the 'Data View Tools: Full Table Export' CJA Product Profile permission
Functionalities:
- Export of processed CJA data to supported cloud accounts
- Inclusion of dimension persistence, attribution, and sessionization
- Send now (one-time) or recurring export options: Daily, Weekly, Monthly(by day of the week or day of the month), and Yearly (by day of the month or specific date)
Benefits:
- Access to fully processed CJA data
- Incorporates advanced CJA features such as calculated metrics, filters, attribution, data view report-time transformations, and derived fields
- Flexible export scheduling to Azure, Amazon S3, Google Cloud Platform, or Snowflake
Considerations:
- Limited to CJA-processed data for up to 5 dimensions and 5 metrics per report
- Unsupported features are automatically excluded from full-table export
- Unsupported components need to be removed before initiating a full-table export
Conclusion
In conclusion, AEP and CJA provide a diverse range of data egress options, each tailored to meet specific organizational needs and technical expertise. From the user-friendly CJA Workspace UI to the powerful AEP Query Service, these tools offer both versatility and depth in data extraction and analysis.
For SQL-savvy users, the AEP Query Service provides a flexible approach to exploring AEP data. Meanwhile, the CJA SQL BI Extension acts as a bridge, connecting CJA's sophisticated analytics with popular BI and visualization tools. This integration allows organizations to seamlessly incorporate CJA insights into their existing BI and visualization solutions.
The CJA Analysis Workspace UI stands out for its intuitive interface, making it accessible to users across various backgrounds. Its drag-and-drop functionality allows for quick creation of custom visualizations and reports. The Analysis Workspace UI's flexibility in combining different data dimensions and metrics fosters a culture of data exploration and sharing of CJA insights within organizations.
For Excel users, CJA ReportBuilder offers a familiar environment for creating custom reports directly in Excel. This tool integrates CJA's analytics capabilities with Excel's spreadsheet functionality, allowing users to leverage their existing Excel skills while harnessing the power of CJA. ReportBuilder's automatic data refresh and email report scheduling features make it a strong choice for regular reporting needs, saving time and ensuring consistent data presentation across teams.
When selecting the optimal egress method, organizations should consider their specific data requirements, team expertise, preferred output format, and integration needs. Key factors include data processing requirements, egress volume limitations, and the necessity for AEP raw or CJA-processed data. Success depends on understanding the strengths and limitations of each egress option and aligning them with your organization's unique data strategy and business objectives. Through thorough evaluation of these options, companies can make informed decisions that enhance their data strategy, streamline operational processes, and drive business success.
