Skip to main content
Level 3
Oct 29, 2026 10:52 AM UTC
New

Destination - add data to the manfest

When exporting data to a JSON file, the accompanying manifest control file currently includes the file SIZE in bytes. However, file size alone is not a reliable validation control for successfull data ingestion, as it doesn't verify whether all records were actually processed and loaded correctly.

Feature Request: We kindly request adding a record count (total number of records) field to the manifest file.

Why this is essential:

  • Reliable Ingestion Validation: A record count provides a definitive checksum to ensure no data was truncated, dropped, or corrupted during export and ingestion pipelines.
  • Automated Quality Control: Data engineering teams can easily automate reconciliation scripts by comparing the source record count against the ingested records.

Adding this metric will greatly improve data reliability and troubleshooting workflows.