How to count your dataset before a buyer asks its size

State the unit, which records it covers, extraction date and exclusion rule with every count. Reconcile raw rows to distinct records and retained records, and report files and bytes separately.

On this page
How to count your dataset before a buyer asks its size

Give every number a unit

“We have a million records” does not tell a buyer what you counted. You may mean messages, cases, rows, files or customers. A case and its replies are different units; adding case and message counts produces a total with no clear meaning.

The World Bank's microdata preparation guide separates record-count checks, unique-identifier checks and duplicate checks. That archive guidance helps distinguish checks you can use when preparing a business-data inquiry.

Start with the data-summary guide for the wider inventory. Use the sheet here to make one volume statement reproducible.

Build a count sheet someone else can rerun

For each figure, retain the source system, export identifier, extraction time and counting rule. Use a stable record identifier where one exists. If the same identifier can appear in several systems, include the system in the rule used to distinguish records.

Count field / What to record
Count fieldWhat to record
UnitCase, message, order, line item, file or another named unit
Records coveredWhich system, dates and categories the number covers
Raw quantityRows or objects present before the stated duplicate and exclusion rules
Duplicate ruleWhich fields identify the same record and how conflicting versions are handled
ExclusionsNon-overlapping groups removed from the count, or an explicit note that groups overlap
Retained quantityWhat remains after the documented rules
How to run the count againSaved query or script version and the export it ran against

Keep file count and byte size in their own fields. W3C's DCAT 3 specification treats the dataset and a supplied file representation separately, including that file's byte size. A ZIP file's size describes that package; it does not establish the number of distinct business events inside it.

A simple helpdesk count

Suppose a helpdesk export contains 1,000 case rows. These round numbers are hypothetical and show the counting method.

Step / Count
StepCount
Start with the exported case rows1,000
Remove 100 duplicate rows using the system and case IDs900
Exclude 200 test or restricted cases, counting each excluded case once700

The volume statement is now “700 cases after removing duplicate rows and the stated exclusions.” Count linked messages separately and name that unit when reporting it. Keep file count and byte size separate, too.

If exclusion categories overlap, count each excluded record once. You can also remove groups in stages, with each step counting only records still present.

Explain why two counts differ

A dashboard and an export may cover different dates, timezones, deleted records or archived statuses. Put the reason beside the discrepancy. If you cannot explain the difference, keep both figures labeled and say the counts have not been reconciled.

Do not quietly change the unit to make figures match. A message count cannot validate a case count. An order count cannot validate a line-item count. Missing identifiers should appear as a documented gap rather than a confident distinct-record total.

Save the counting rule with the result

Save the sheet with the export reference and counting instructions. Add the resulting volume statement to the dataset manifest, retaining the unit and exclusions in the description. The rights checklist addresses the separate question of what you may share.

Sources and limitations

2. Gathering and Preparing the Data Set | Guide for Microdata Archivists Checked 2026-11-15

World Bank · Primary source

Verify Record Counts

Archival microdata documentation guidance adapted to an original business-records preparation worksheet; not AI buyer requirements.

View source reference ↗
Data Catalog Vocabulary (DCAT) - Version 3 Checked 2026-11-15

W3C · Primary source

Property: byte size

Vocabulary for metadata description; using its concepts does not require a complete DCAT catalog or prove data rights or buyer acceptance.

View source reference ↗