Introducing Data Distiller Dashboards 2.0

[#1]

Introduction

In this tutorial, we will explore how to create powerful, enterprise-grade dashboards that rival the best-in-class dashboards found in traditional business intelligence (BI) tools. The goal is to demonstrate that you don't need to rely on expensive BI vendors or invest extra resources for users to build and consume insights effectively. By leveraging SQL as the foundation for creating these charts, we can unlock the full potential of your data without the need for additional software or tools.

The tutorial is designed with two key audiences in mind: the data team and the business users. The data team will focus on building foundational charts, ensuring that they include the necessary filters and dimensions to provide flexibility. This allows business users to perform deep, drill-down analyses and gain actionable insights directly from the dashboards. This approach streamlines the workflow, enabling teams to quickly create interactive and insightful dashboards without having to leave the Data Distiller environment.

By the end of the tutorial, you'll have the skills to create sophisticated dashboards that empower business users to make data-driven decisions—without the complexity and cost typically associated with external BI platforms.

Data Distiller Dashboards UI and backend have undergone a major revamp.

Data Distiller Accelerated Store

Instead of writing queries directly against the AEP Data Lake, you can use the accelerated store, which features a high-performance engine designed for faster dashboard queries. This engine significantly enhances the speed and efficiency of data retrieval, ensuring that dashboards load quickly without compromising data accuracy. Best of all, this capability is included as part of the Data Distiller license, making it a seamless, cost-effective solution for boosting query performance in your dashboards

 

Audience analysis including overlaps and identity composition

CRUD Support in Accelerated Store

A new MERGE INTO and UPDATE/DELETE syntax enables you to update/delete records between your source table and the target table in the Accelerated Store that allows you to do low volume surgical deletes.

REST API Support in Accelerated Store

You can leverage REST APIs to post queries from any application and retrieve the results as JSON. This allows for seamless integration of Data Distiller’s querying capabilities into your own applications, providing flexible and automated access to data insights for reporting, analysis, or further processing.

 

Conceptual mock of a Data Distiller Data Model

Data Distiller Data Models

A Data Distiller Data Model, much like a reporting star schema, organizes customer and campaign data for efficient insights. The fact table can store key metrics like campaign impressions, clicks, or purchases, while dimension tables provide additional context, such as customer demographics, product categories, or marketing channels. This structure enables marketers to quickly analyze performance across various dimensions, track campaign effectiveness, and identify trends, offering a clear view of customer behavior and campaign ROI. The star schema of the data model enhances query performance, making it ideal for marketing dashboards and ad hoc reporting.

A normalized relational data model is optimized for representing data in a compact and efficient way by minimizing redundancy and organizing information across multiple related tables. Each piece of data is stored only once, reducing storage needs and avoiding duplication. This design allows for updates, like changes in a dimension (e.g., customer data), to be made in one place without reprocessing the entire dataset. As a result, updates are more efficient, saving processing time and ensuring consistency across related data in different tables.

Data Distiller Data Views

Data Distiller Data Views addresses the usability challenges of normalized data models by enabling data engineers to create flat views tailored for marketing users. These flat views present the underlying data in a simplified format, making it easy for users to build dashboards or perform ad hoc analysis without dealing with the complexity of the normalized model. This approach ensures that marketing teams can work with familiar, user-friendly data structures, streamlining their analysis while maintaining the benefits of the underlying model's efficiency and flexibility.

Data Distiller Charts

A Data Distiller Dashboard typically includes several key visualizations: Big Numbers (KPIs) to highlight critical metrics like total web traffic, Line Charts to track trends over time (e.g., traffic or sales), Bar Charts to compare categories (like products or regions), Donut Charts to show proportions of a total, and Tables to present raw, detailed data for deeper analysis. These elements together provide a comprehensive view of performance and insights, enabling quick decision-making and further investigation

Download as PDF

You can easily download the entire dashboard as a single-page PDF, making it convenient to share with stakeholders. This feature is especially useful for presenting key insights in a visually organized format, allowing stakeholders to review and understand the data without needing access to the dashboard itself. It ensures consistency in reporting and facilitates clear communication, whether for meetings, presentations, or email sharing.

ViewMore and ViewSQL

In a Data Distiller Dashboard, each chart can be expanded to view the underlying data, displayed in a table format with pagination, making it easy to explore like an Excel file. You can also download the data as a CSV for further analysis. The ViewSQL feature reveals the SQL query used to generate the chart, allowing users to reuse the query in other charts or add it to their private LLM for advanced personalization and modeling. This flexibility enhances both data exploration and customization.

Data Distiller Drilthroughs

Data Distiller Drillthroughs are a feature that allows users to explore data more deeply by clicking on a chart within a dashboard and being directed to another detailed report or dashboard. The purpose of a drillthrough is to provide context and further insights without overwhelming the primary dashboard with excessive details.

For example, if a user clicks on a marketing leads figure in a regional dashboard, a drillthrough might show detailed transaction records or performance metrics specific to that region, helping users investigate trends or anomalies efficiently.

SQL Authoring of Charts

One of the main challenges with using drag-and-drop interfaces to create charts is the lack of flexibility when it comes to defining custom metrics. While these interfaces are convenient, they often fall short in handling more complex calculations. On the other hand, SQL provides unmatched flexibility in metric definition. For instance, if you want to visualize a trailing 30-day average using a rolling window for each date, achieving this in a typical drag-and-drop dashboard interface would be nearly impossible. You would likely need to recompute the entire metric at the ETL layer instead.

Our goal of building this feature was to unleash the flexibility of SQL at the chart authoring layer.

Data Distiller Global Filters

We’ve adopted an innovative approach to filter design by allowing filters to be created at the dashboard level while giving you full control over how they’re applied within individual charts. This flexibility enables sophisticated filter logic, where both local filters and the chart’s context work together to determine how the filter impacts the data displayed. This advanced filter design ensures greater customization and precision, allowing you to tailor the behavior of each chart based on specific business needs.

Data Distiller Date Filters

An advanced date picker-style filter, offering both date range selection and preset options, can be applied to charts within the dashboard. This feature allows users to quickly customize date-based filters, enhancing flexibility and precision in data analysis.

Pushdown Filters in Drillthroughs

When a drillthrough is applied on a dashboard, the global filter can also be applied to the child elements, provided they are connected to the same filter. This ensures that the global filter will influence all related child charts and reports within the dashboard, even when navigating deeper into the data through drillthrough actions. This setup maintains consistency across visualizations and enhances the interactivity of the dashboard.

BI Connectivity to Accelerated Store

You can integrate your preferred BI tool with the data models stored in the Accelerated Store, enabling seamless access to high-performance, optimized data. This allows users to leverage the power of the Accelerated Store's query engine while continuing to work within familiar BI environments for dashboard creation, reporting, and analysis. The flexibility of this integration ensures that businesses can take advantage of both their existing BI tools and the advanced capabilities of the Accelerated Store without sacrificing speed or efficiency.

Federation Across Accelerated Store and Data Lake

Data Distiller enables you to read and write datasets across both the AEP Data Lake and the Accelerated Store. From the query engine's perspective, when using the Data Distiller Query Pro Mode Editor (or a third party client), you can seamlessly join tables from both stores and persist the results across them. To ensure the query engine writes to the Accelerated Store rather than the AEP Data Lake, you need to qualify the data model using specific parameters, such as WITH (TYPE=QSACCEL, ACCOUNT=acp_query_batch). This syntax signals to the system that the target is the Accelerated Store.

Read the Tutorial

Here is the tutorial