Measurement
Part of PR agencies measurement: reject AVE, keep evidence levels apart
Three PR measurement mistakes in the data: permissions, lineage, benchmarks
Three data-side PR agencies measurement mistakes for England: permission-related data loss, figures without lineage and incompatible benchmarks.
PR agencies measurement mistakes in the data are easy to miss because the chart still looks tidy. Three of them sit upstream of any dashboard: permission-related data loss, a figure without lineage and benchmarks that cannot be combined. The checks below are non-ranked and make no claim about how often agencies make them.
What to take away
- Analytics can exclude people or events for permission and technology reasons, so never fill an unknown with zero.
- A dashboard figure needs reproducible lineage covering source system, field, query version, refresh time and owner.
- Two figures with the same label may cover different geographies, channels, periods, samples or denominators.
- Compare population, unit, collection and uncertainty line by line before combining any figures.
- Keep England, UK and global evidence separate, and publish the gap rather than an average when compatibility fails.
Published method
- Scope and date:research date 6 September 2026; PR and communication measurement decisions for an England-operating buyer, using UK sources where rules or public-sector methods are UK-wide.
- Inclusions and exclusions:a distinct error with an authoritative regulator, government or official-statistics record and an observable repair test. Excluded: supplier rankings, campaign results, prices, prevalence estimates and anonymous complaints.
- Ranking and conflicts:none; sequence does not indicate frequency, cost or severity. No agency, platform, monitoring supplier or affiliate paid for inclusion.
The three checks side by side
Each mistake has one question to ask before a figure is published and one repair or hold rule. The questions can be answered from the buyer's own records, without any supplier ranking or campaign result.
| Mistake | Question before publishing | Hold or repair |
|---|---|---|
| Hidden permission-related data loss | Who could not be counted, and why? | State population, route, permission state and excluded share |
| Figure without lineage | Can the value be reproduced from its source? | Pause the card until lineage is documented |
| Incompatible benchmarks | Do population, unit, collection and uncertainty match? | Publish the gap rather than an average |
1. Hiding permission-related data loss
A figure is only as complete as the collection behind it. The ICO says PECR applies to any technology that stores or accesses information on a user's terminal equipment, which includes cookies and tracking pixels.
Analytics may exclude people or events because technologies were not used, permission was absent, settings changed or a browser blocked collection. The ICO's final storage and access technologies guidance covers relevant cookies, pixels and similar tools under PECR and UK GDPR. It does not promise complete measurement.
Repair: state the eligible population, collection route, permission state, excluded share where knowable and any modelling. Never fill an unknown with zero. A blank is honest, while a zero is a claim. Mapping media data flows shows where the collection points sit.
2. Publishing a figure without its lineage
A dashboard value can outlive the query that made it. Require the source system, field, query or model version, refresh time, owner and correction history, and pause the card when lineage cannot be reproduced.
NCSC audit-information guidance supports checking log availability, format and retention for cloud services. This is a security control, not a statistical guarantee, so it cannot stand in for the lineage record of a PR figure.
Hold test: can another analyst rebuild the value from the stated source and version? A measure record holds the owner and correction history, the metric dictionary holds the definition, and the reporting dashboard should show both.
3. Combining incompatible benchmarks
Two figures with the same label may cover different geographies, channels, periods, samples or denominators. Compare population, unit, collection and uncertainty line by line, as set out in benchmark research, before any figures are combined.
The UK Statistics Authority's 2020 letter on COVID-19 testing figures is a public-sector example. It said it should be clear whether the target reflected testing capacity, tests administered, test results received or the number of people tested.
ONS quality and methodology information explains that its official-statistics notes cover relevance, accuracy, timeliness, accessibility, coherence and comparability. It does not validate a private PR benchmark.
Repair: keep England, UK and global evidence separate. If compatibility fails, publish the gap rather than an average.
Hold the list for review
The list remains on publication HOLD. A named measurement reviewer, working to the review method, must test these checks against the buyer's real definitions, data rights and decision record before release. The measurement system guide shows how those records fit together.
Before you act
- State eligible population, collection route, permission state and excluded share.
- Replace every unknown zero with a stated gap.
- Require source system, field, query version, refresh time and owner.
- Pause any card whose lineage cannot be reproduced.
- Compare population, unit, collection and uncertainty before combining benchmarks.
- Keep England, UK and global evidence in separate fields.
Common questions
Why must an unknown never be filled with zero?
Analytics can exclude people or events for permission and technology reasons. A zero claims that nobody was missed, so state the eligible population, collection route and excluded share instead.
Does NCSC audit-information guidance make a PR figure reliable?
No. It supports checking log availability, format and retention for cloud services. That is a security control, not a statistical guarantee, so the figure still needs its own lineage record.
Why can two figures with the same label not be combined?
The label often hides differences in population, unit or collection method. ONS quality notes cover official statistics; they do not certify a private benchmark. If compatibility fails, report the two figures side by side instead of averaging them. Note which one covers England.
