Imagine this
A computer can make a short check-number from a file. If the file changes, the number should change too. But the number cannot tell us whether the file tells the truth.
THE FIELD GUIDE / 033 · DATA INTEGRITY AND PROVENANCE
An unchanged record.
A true statement?
A report can be unchanged, correctly signed and still contain a mistake. Ask where it came from, what changed, who controls its signing key and what evidence supports its contents.
Change one character ↓A very simple version
For curious readers: how it could help and what the evidence says
How it could help people
Today, a file’s check-number or digital signature can help show whether saved data changed. A separate linked record can describe how someone changed the data, if each step is written down.
Evidence and limits
Integrity checks do not prove a source claim, protect a compromised key or decide who should have access.
Read-aloud starter; not yet tested with children. The fuller science, data, figures and sources follow below.
Ten copies can share one source
Imagine one instrument reading copied into a report, translated and reposted. Several records now exist, but they depend on the same initial reading. Repetition spreads information; it does not create independent observations.
Record acquisition origin and distinguish original, copy, OCR transcription, translation and edited derivative. An original timestamp or a consistent internal chronology does not supply independent corroboration. “Sent”, “received” and “independently observed” describe different evidence.
The W3C provenance data model helps represent entities, activities and responsible agents. A well-formed history is useful, but its format cannot certify truth.
FABRICATED VERSION GRAPH · STRUCTURE ONLY
Count records without counting them as witnesses
A is an asserted original observation record. B is a report derived from A; T is a translation of B. Add copies of B, then introduce a structural problem. The list shows the current graph; the illustration above is a separate example.
Visible record entries
Declared roots when structure is interpretable
Fixture context: these labels do not include actual raw observations, calibration files or full document pages. Passing the chosen structural checks does not make those missing materials available. A root is an asserted starting point, not an independently verified witness.
A missing parent makes ancestry unresolved. An evidence field can record a reason for claimed independence, but cannot verify that reason. Cycles are flagged in this chosen immutable-version ancestry schema, not banned from every possible provenance relationship. The duplicate-ID case is an added UI check beyond the four historical graph cases.
Keep gaps and transformations visible
A missing page may contain a condition that changes a conclusion. Adjacent pages are not automatically one original document. Preserve uncertain attribution and later annotations separately; label any reconstruction as conjecture. A complete byte-for-byte copy can still be an incomplete document.
For an instrument record, retain raw samples plus units, acquisition settings, timestamps and timebase, calibration, processing steps and software versions. A graph or decoded sentence is a derived product: it cannot restore information discarded earlier. Raw data can also be inaccurate or fabricated; preservation enables scrutiny rather than replacing it.
REAL SHA-256 · LOCAL UTF-8 TEXT
A fingerprint checks bytes
Edit this invented note. Your browser calculates SHA-256 over the UTF-8 bytes of the text field, without trimming or Unicode normalization, and compares it with the unchanged demonstration note supplied by this page. No text is submitted to a server by this lab.
Demonstration reference fingerprint
Current text fingerprint
Changing one character normally changes the fingerprint. This checks typed text, not an uploaded document or screenshot. Textarea line endings are represented by the browser; the resulting UTF-8 bytes are what this lab hashes.
The reference comes from this same page, so it is a teaching baseline, not independent authentication. In a real comparison, the earlier fingerprint must come from a trusted source. If someone replaces both the file and an untrusted reference, matching fingerprints no longer establish that the original was preserved. Hashes also do not identify the author or prove the statement true.
A signature needs a trusted key
A digital signature can be checked with a public key. The corresponding signing key must be protected, and the verifier needs a reliable basis for associating the public key with the intended person or organization. A copied visual seal does not provide this mathematical check.
Our offline study signs a manifest listing file fingerprints and byte lengths. Checking the bundle involves two jobs: verify the signature on that manifest, then compare the listed files with it. A changed file can fail its fingerprint check even while the unchanged manifest's signature remains valid. RFC 8032 specifies Ed25519.
An exact copy can verify. A screenshot, OCR correction or edited transcription is a new byte sequence and should be recorded as a derivative. A signature does not establish content truth, an independently trusted date or permission to publish.
VISIBLE STATE DEMONSTRATION · NO CRYPTOGRAPHY IN THIS WIDGET
Try the same response twice
A fixed response can be recorded and repeated. A challenge-based exchange can instead require a response bound to a fresh challenge and a particular action. This widget illustrates the state checks with visible, predictable numbered challenges; it does not authenticate anyone.
Unused challenge and bound action
Captured response fields
Issue, capture and submit: the first matching submission is accepted. Submit again: it is rejected. Issue a new challenge and submit the old response without capturing a new one: it is rejected. Changing the requested action also invalidates a response bound to a different action.
Every submission consumes the pending challenge, including a failed one. This deliberately simple policy can let failed attempts disrupt a session. Issuing a new challenge replaces the previous pending one. Reset discards all state and restarts numbering. There is no expiry, persistent state, rate limiting, protected credential or relay defense.
The downloadable study below executes real HMAC-SHA256 separately. HMAC uses a shared secret; it is not a publicly verifiable signature. Freshness and action binding come from the protocol and its state, not from hashing alone. RFC 2104 explains HMAC.
No record is actually read or replaced by these buttons, and they do not change the website's authentication or permissions.
Permission is a separate question
A voice recording saying yes is a record of a statement. By itself, it establishes neither who spoke, whether it was edited or replayed, whether agreement was freely given nor whether it still covers the intended action.
Describe purpose and scope, provide a clear way to decline and offer a non-voice option. Keep current authorization separate from historical records. Authentication does not automatically permit every use, and withdrawing permission does not necessarily erase every historical audit record. This chapter makes no legal consent determination. NIST distinguishes authentication intent and replay resistance.
A controlled original and a shareable redacted derivative can coexist. Preserve the transformation history and the applicable access restrictions. This page collects no voice or identity evidence.
What could this improve?
Today: researchers can hand over data with a usable history, editors can distinguish corrections from originals, and readers can trace multiple claims to a shared source. Teams can also check whether a signed data bundle changed.
A possible next step: combine interoperable provenance, integrity checks and reliable key-to-organization records so evidence is easier to inspect. Agreed metadata, access rules and independent review remain necessary; such a system would not automatically judge the truth of every statement.
A FAMILIAR EXAMPLE
What can an envelope seal tell you?
A sealed and stitched envelope asks a practical question: can the recipient notice that it may have been opened? A broken seal or changed stitching can be a reason to investigate, but ordinary handling damage can also create a false alarm. Assessing this proposal requires a record of the original condition and clear signs to check.
Opening detection: look for evidence of a change. Authentication: establish who sent it. Confidentiality: protect the contents from being read. These are separate properties: an apparently intact envelope proves neither the sender’s identity nor that the contents stayed private.
A concrete use: inspect a received parcel against its documented condition before accepting it. A possible study: compare recorded examples of ordinary handling and opening, and count both missed changes and false alarms. No detection rate has been measured for the proposed sealed-and-stitched arrangement. Read the open idea and its missing evidence.
REPRODUCIBLE CASE STUDY · FABRICATED RECORDS
Expected outcomes, not a security percentage
18 expected and actual cases CSVOutcome matrix PDFFabricated provenance graphsPublic signature and harmless fixturesMethods and limitsReproduction scriptRuntime and parametersExecution checks
| Case family | Observed outcome | What remains unproved |
|---|---|---|
| Repeated fixed response | Both first use and replay accepted | The insecure baseline provides no freshness |
| Fresh HMAC challenge | Valid first response accepted; replay, stale response, wrong key and changed context rejected | Production protocol security and deployment quality |
| Signed fabricated bundle | Untouched and exact copy accepted; five alterations or key/manifest mismatches rejected | Author identity, truthful content, trusted time and permission |
| Version lineage | Valid chain accepted; unsupported independence, loop and missing parent flagged | Authenticity and factual independence of records |
What ran?
A clean rerun of the existing case families executed SHA-256, HMAC-SHA256 and Ed25519 on fabricated English fixtures. HMAC uses a fresh 32-byte demonstration key and 16-byte challenges, with compact JSON [nonce, context]. The verifier consumes a challenge before checking context and the authentication code. The failed-attempt consumption rule is also checked separately, not counted as a nineteenth trial.
Ed25519 signs canonical UTF-8 JSON with sorted keys and compact separators, containing each listed file's SHA-256 and byte length. Untouched bytes and an exact copy pass. A changed byte, generated raster-page replacement and edited transcription fail the file-integrity check. A wrong public key and modified manifest bytes fail signature verification. These are seven selected cases, not an exhaustive security test.
Private signing keys and shared secrets are generated in memory and never saved in the study files. The exported fixtures contain the public key, signature, manifest and harmless data needed to check the example. Fresh keys make exact cryptographic bytes vary between runs while expected outcomes remain reproducible.
The provenance study uses its declared version-lineage schema. Checking an evidence field's presence does not investigate its contents. These four hand-selected cases are structural checks, not archival findings.
Reproduce and interpret
Run the downloaded script with Python, cryptography, Pillow and matplotlib. Runtime versions and the generator hash are recorded. Historical scripts and results were preserved; the public export uses new fabricated fixtures. The typed-text browser hash is a separate exercise, and the visible replay widget is a state model rather than the offline cryptographic implementation.
The bundle demonstration checks listed entries. It is not a hardened validator for hostile archives, extra unlisted files, duplicate paths or every malformed manifest. No case count supplies a security probability. No new archival discovery, identity verification, consent study or real signal acquisition is claimed.