Data integrity
« Back to Glossary Index
Data integrity is the assurance that data remains accurate, complete and consistent across its entire lifecycle — from the moment it is created, through every transfer, transformation and storage step, to the point it is used or deleted. When integrity holds, a value means the same thing wherever it appears and has not been altered in transit.
It is a broader idea than correctness at the point of entry. Data can be entered correctly and still lose integrity later through a failed load, a partial update, an unlogged manual fix or a transformation that silently drops records.
What Is Data Integrity?
Data integrity has two commonly used senses, and conflating them causes confusion in practice.
The database sense refers to structural rules enforced by the system itself — primary keys preventing duplicates, foreign keys preventing orphaned records, constraints preventing invalid values. These are guarantees the database makes and will not permit to be violated.
The information governance sense is broader: whether data is trustworthy and defensible across its whole life, including whether changes are traceable and whether the record is complete. This is the sense regulators use, and it extends well beyond what any database engine enforces.
Most enterprise conversations about data integrity mean the second while relying on the first. The gap between them — everything that happens in pipelines, spreadsheets and manual corrections between systems — is where integrity is usually lost.
Data Integrity vs. Data Quality vs. Data Security
These three overlap and are routinely used interchangeably, which obscures what each actually addresses.
Data quality asks whether data is fit for its intended use — accurate, complete, timely, consistent and valid. It is assessed against business requirements, and the answer can differ by use case.
Data integrity asks whether data has remained intact and unaltered through its lifecycle. A dataset can have poor quality while having perfect integrity: the values were always wrong, and nothing has changed them since.
Data security asks whether data is protected from unauthorized access or modification. Security is a means of preserving integrity, but integrity also fails through entirely accidental causes — a truncated load, a mismatched join — that no security control would catch.
In practice the three are managed together, which is why data quality and data governance programs typically own integrity controls rather than treating them as a separate discipline.
Types of Data Integrity
Entity integrity ensures every row is uniquely identifiable, enforced through primary keys. Without it, duplicates accumulate and counts become unreliable.
Referential integrity ensures relationships between tables stay valid — an order cannot reference a customer that does not exist. Loss of referential integrity is a frequent side effect of migrations where tables move independently.
Domain integrity ensures values fall within permitted ranges, formats and types. It is what prevents a date of birth in the future or a status value nothing recognizes.
User-defined integrity covers business rules specific to the organization — a discount that cannot exceed a threshold, a claim that cannot be settled before it is validated. These rarely exist as database constraints and are usually enforced in application or pipeline logic, which is why they are the easiest to lose.
What Threatens Data Integrity
Pipeline failures. A load that fails partway leaves a partial dataset. If the failure is not detected, downstream consumers report on incomplete data without knowing it — the most common and most damaging integrity failure in analytics estates.
Transformation errors. A join that silently drops unmatched records, a type conversion that truncates, a filter with an off-by-one boundary. None of these raise errors; they simply produce fewer or different rows.
Manual intervention. Corrections applied directly in production, or figures adjusted in a spreadsheet between the system and the report. Even when correct, an unlogged change destroys traceability.
Migrations. Moving platforms is when integrity is most at risk, because constraints do not always transfer with the data and reconciliation is often done on aggregates that hide offsetting errors.
Concurrency and sync issues. Two systems updating the same record, or a replication lag that leaves consumers reading stale state while believing it current.
How to Maintain Data Integrity
Enforce constraints at the source. Keys, foreign keys and validation rules in the database prevent whole categories of problem from ever entering the estate. Cleaning downstream is always more expensive than preventing upstream.
Validate inside pipelines, not after. Row counts, checksums, referential checks and range tests should run as pipeline steps with the ability to halt the load. Validation that runs only after publication tells you about a problem your users have already seen.
Reconcile at row level. Compare source and target record by record, particularly during migration. Aggregate totals conceal errors that cancel out.
Maintain lineage. Knowing where a value came from and what transformed it is what makes an integrity problem diagnosable rather than merely visible.
Log every change. Audit trails covering who changed what and when are the difference between an explainable correction and an unexplained discrepancy.
Control manual access to production. Direct edits should be rare, approved and logged. Most untraceable discrepancies begin as a well-intentioned manual fix.
Data Integrity in Regulated Industries
In life sciences, financial services and healthcare, data integrity is a compliance obligation with defined expectations rather than an internal quality goal.
Life sciences regulators apply the ALCOA principles — data should be attributable, legible, contemporaneous, original and accurate — with later extensions covering completeness, consistency, endurance and availability. Financial reporting regimes require controls over the accuracy of reported figures and evidence that those controls operate. Healthcare regimes impose integrity and auditability requirements on patient records.
The practical consequence is that audit trails, version history and demonstrable controls stop being good practice and become evidence. Systems that cannot show how a number was produced create a compliance exposure independent of whether the number was correct.
Integrity controls matter most where records are shared across systems, which is the problem master data management for enterprise AI sets out to solve.
Frequently Asked Questions
Data integrity is the assurance that data stays accurate, complete and consistent across its whole lifecycle — through creation, transfer, transformation and storage — so a value means the same thing wherever it appears and has not been altered without trace.
Data quality asks whether data is fit for its intended use. Data integrity asks whether it has remained intact and unaltered through its lifecycle. Data can have perfect integrity and poor quality — the values were always wrong and nothing has changed them.
Security protects data from unauthorized access or modification and is one means of preserving integrity. But integrity also fails through accidental causes such as a truncated load or a mismatched join, which no security control would prevent.
Entity integrity ensures each row is uniquely identifiable. Referential integrity keeps relationships between tables valid. Domain integrity restricts values to permitted ranges and formats. User-defined integrity enforces organization-specific business rules.
Partial pipeline failures, transformation errors such as joins that silently drop records, unlogged manual corrections, platform migrations where constraints do not transfer, and concurrency or replication issues that leave systems out of sync.
Enforce constraints at source, validate inside pipelines with the ability to halt a bad load, reconcile at row level rather than on totals, maintain lineage, log every change, and tightly control direct manual edits to production data.
ALCOA is a life sciences regulatory principle requiring data to be attributable, legible, contemporaneous, original and accurate. Extended versions add complete, consistent, enduring and available.
Models and reports inherit whatever integrity problems exist upstream. A partial load or a silently dropped join produces confident output based on incomplete data, and because nothing errors, the result looks valid while being wrong.
