CoordTransform Cadastral & High-Precision Python

Generating Audit Hashes for Transformation Batches

A cadastral deliverable is only tamper-evident if any change to its coordinates or transformation parameters is provably detectable, which is the integrity role inside compliance report generation for agency submission and, more broadly, of Batch Transformation & Automation for Cadastral Coordinate Pipelines. This page builds the deterministic SHA-256 audit hash that makes a transformed batch both tamper-evident and reproducible: a single digest computed over a canonical serialisation of the inputs, the transformation parameters, and the outputs, such that any independent party re-running the same three ingredients arrives at the identical 64-character hex string.

Canonical audit-hash construction Three inputs stack on the left: input coordinates canonicalised to little-endian float64 bytes, transformation parameters serialised as JSON with sorted keys, and output coordinates canonicalised the same way. Each feeds, in a fixed order, into a single SHA-256 hasher in the centre, which emits one 64-character hexadecimal digest on the right. The fixed order and canonical form are what make the digest reproducible. inputs → bytes <f8 little-endian params → JSON sorted keys · UTF-8 outputs → bytes <f8 little-endian SHA-256 ordered update() 64-hex digest

Figure — inputs, parameters, and outputs enter one hasher in a fixed order and canonical form to yield a reproducible digest.

Why Canonicalisation, Not Just Hashing

SHA-256 is deterministic by construction: identical bytes always yield the identical digest. The difficulty is never the hash function — it is guaranteeing that “the same batch” produces the same bytes on a different machine, on a different day, under a different NumPy build. Two obstacles dominate. First, the textual representation of a float is not unique: repr(0.1) and a rounded string of the same value differ byte-for-byte, and JSON serialisers do not agree on float formatting. Second, a NumPy array’s in-memory bytes depend on its dtype and the machine’s endianness, so arr.tobytes() on a little-endian x86 host differs from the same logical array on a big-endian host.

Why a coordinate must be canonicalised before it is hashed Five ways the same value reaches a hash function. The Python repr of a float varies with the platform and the interpreter version. A format string with default precision silently truncates. Scientific notation differs in exponent width across libraries. A locale-formatted string can use a comma as the decimal separator. Only a fixed decimal string at a stated precision, with a stated rounding mode, hashes to the same digest everywhere, which is the whole point of an audit hash. repr(x) varies platform and version dependent f"{x}" truncates default precision is not a contract f"{x:e}" varies exponent width differs locale format varies decimal comma in some locales Decimal, 8 dp, half-even stable the only hashable form

Figure — five representations of the same coordinate; only one of them hashes stably.

Canonicalisation removes both degrees of freedom. Coordinate arrays are cast to an explicit little-endian float64 dtype (<f8) before .tobytes(), fixing width and byte order regardless of platform. Parameters are serialised as JSON with sorted keys and compact separators, and any float parameter is formatted with a fixed precision so its textual form is stable. The resulting byte stream is a canonical form: a well-defined function of the batch’s meaning, not of its incidental encoding. Only then does hashing become reproducible in the sense an auditor needs.

Complete Runnable Implementation

The module below hashes a batch from three ingredients. It exposes a small canonicaliser for arrays, a fixed-precision float formatter for scalar parameters, and the top-level hash_batch that assembles them in a fixed order.

from __future__ import annotations

import hashlib
import json
from typing import Any

import numpy as np


def canonical_array_bytes(arr: np.ndarray) -> bytes:
    """Return platform-stable bytes for a coordinate array.

    The array is made contiguous and cast to little-endian float64 (`<f8`)
    so its byte image is identical on any architecture. Casting also
    normalises int or float32 inputs to one canonical width.
    """
    canonical = np.ascontiguousarray(arr, dtype="<f8")
    return canonical.tobytes()


def canonical_params(params: dict[str, Any], float_ndigits: int = 9) -> bytes:
    """Serialise transformation parameters to a canonical UTF-8 byte string.

    Keys are sorted, separators are compact, and every float is rounded to a
    fixed number of decimal places so its textual form cannot drift between
    platforms or serialiser versions.
    """
    def _normalise(value: Any) -> Any:
        if isinstance(value, float):
            return round(value, float_ndigits)
        if isinstance(value, dict):
            return {k: _normalise(v) for k, v in value.items()}
        if isinstance(value, (list, tuple)):
            return [_normalise(v) for v in value]
        return value

    normalised = _normalise(params)
    text = json.dumps(normalised, sort_keys=True, separators=(",", ":"),
                      ensure_ascii=False)
    return text.encode("utf-8")


def hash_batch(
    inputs: np.ndarray,
    outputs: np.ndarray,
    params: dict[str, Any],
    float_ndigits: int = 9,
) -> str:
    """Deterministic SHA-256 over canonical inputs + parameters + outputs.

    The three ingredients are fed in a fixed order with a length-prefixed
    domain separator between them, so a value cannot migrate across the
    boundary and forge a colliding digest. No timestamp is included, so the
    hash is reproducible from the coordinates and parameters alone.
    """
    h = hashlib.sha256()
    for label, chunk in (
        (b"inputs", canonical_array_bytes(inputs)),
        (b"params", canonical_params(params, float_ndigits)),
        (b"outputs", canonical_array_bytes(outputs)),
    ):
        h.update(label)
        h.update(len(chunk).to_bytes(8, "big"))  # length prefix = domain sep
        h.update(chunk)
    return h.hexdigest()

Parameter Reference

Name Type Units Valid range Notes
inputs np.ndarray coordinate units any shape source coordinates; cast to <f8 before hashing
outputs np.ndarray coordinate units matching batch transformed coordinates; canonicalised identically
params dict[str, Any] — JSON-serialisable operation code, grid, method — the reproducibility metadata
float_ndigits int decimal places 6–12 typical fixed float precision; 9 dp ≈ sub-mm at metre scale
return str — 64 hex chars the audit digest; recompute to detect any change

Worked Example

Hashing a three-point batch twice — with the arrays rebuilt from scratch — yields the identical digest, which is the whole point of the exercise.

inputs = np.array([[500000.0, 5000000.0],
                   [500100.0, 5000100.0],
                   [500200.0, 5000200.0]])
outputs = np.array([[500000.006, 5000000.004],
                    [500100.005, 5000100.007],
                    [500200.008, 5000200.003]])
params = {"operation_code": "EPSG:1888", "method": "NTv2",
          "grid_file": "us_noaa_nadcon5.tif", "grid_version": "2021.1"}

digest = hash_batch(inputs, outputs, params)
print(len(digest), digest[:12])   # 64  <first 12 hex chars>

# Rebuild the arrays independently; the digest is byte-identical.
again = hash_batch(inputs.copy(), outputs.copy(), dict(params))
print(digest == again)            # True

Validation Check

Assert both properties an audit hash must have: reproducibility across independent reconstructions, and sensitivity to a sub-millimetre change in any output coordinate.

What an audit digest must cover Five inputs and whether the digest must cover them. The canonicalised coordinates must be covered, obviously. The CRS identifiers and epochs must be covered, or the same numbers under a different CRS pass verification. The operation identifier and grid checksum must be covered, or a changed grid is invisible. The row order must be covered, because a reordered file is a different deliverable. The wall-clock timestamp must NOT be covered, or the same batch never verifies twice. In the digest? If omitted Canonical coordinates yes nothing is covered CRS ids + epochs yes wrong CRS verifies Operation + grid hash yes a changed grid is invisible Row order yes a reorder verifies Wall-clock timestamp no nothing verifies twice

Figure — what the digest must cover: leaving anything out makes the hash a decoration.

assert hash_batch(inputs, outputs, params) == digest, "hash not reproducible"

tampered = outputs.copy()
tampered[1, 0] += 0.0005          # a 0.5 mm edit to one easting
assert hash_batch(inputs, tampered, params) != digest, "tamper went undetected"

Common Mistakes

Hashing pretty-printed or default JSON
Calling json.dumps without sort_keys=True lets key order — and therefore the bytes — depend on dict insertion order, and default separators add whitespace that varies. Two logically identical parameter sets then hash differently. Always sort keys and use compact separators for the canonical form.
Relying on float repr instead of fixed formatting
Serialising a raw float leans on the platform's shortest-repr algorithm, which can differ across interpreter versions and locales. Round every float parameter to a fixed number of decimals (here float_ndigits) before serialising so its textual form is pinned.
Ignoring array dtype and endianness
Calling arr.tobytes() on the native array hashes whatever width and byte order the array happens to hold — float32 versus float64, little- versus big-endian — so the digest is not portable. Cast to an explicit <f8 dtype first; that fixes both width and endianness on every platform.
Baking a timestamp into the hashed stream
Including datetime.now() in the digest makes every run produce a different hash, destroying reproducibility — an auditor can never re-derive it. Record the timestamp beside the digest, never inside it; the hash must be a function of the coordinates and parameters alone.

Frequently Asked Questions

Why not hash the output file directly?

You can, and it is a useful integrity check on that file — but it answers a narrower question. A file hash changes when the line endings change, when a column is reordered, or when the same coordinates are exported to a different format, none of which alter the deliverable’s content. A hash over canonicalised coordinates plus the CRS and operation metadata verifies what was computed rather than how it was written down.

Should the timestamp be inside the hash?

No. Include it in the record, next to the digest, but not in the digest itself: if the wall-clock time is hashed, re-running the same batch over the same inputs produces a different digest and the hash stops being able to answer the only question it exists to answer — did this run produce the same result as that one.

Which hash algorithm should I use?

SHA-256 is the sensible default: it is fast, universally available, and its output length is not a burden in a metadata block. Avoid MD5 and SHA-1 for anything that has to stand up years later, not because a cadastral file is a likely target but because explaining why a deprecated algorithm was acceptable is a conversation you do not want to have in a dispute.