Writing Golden Vector Tests for Coordinate Transforms
A golden vector is a coordinate whose transformed value is known independently of your code, and it is the only test that catches the failure mode that matters most in this domain: an operation that silently changed and still returns plausible numbers. This guide, part of testing and CI for coordinate transformation code, covers where to get vectors that are genuinely independent, how to choose the ones worth having, and what tolerance a vector should carry so a failure means something.
What Makes a Vector Golden
Three properties, and dropping any one of them turns the test into a record of current behaviour.
Figure — drop any one of the three properties and the test records behaviour instead of testing it.
Independence. The expected value must come from somewhere other than the implementation under test: a published control coordinate and its published transformed value, an agency’s worked example, a second independent implementation, or a hand computation. A fixture generated by running the code captures whatever the code did on the day, including its bugs.
Pinned context. A transformed coordinate is only reproducible against a stated operation, a stated PROJ version and stated grid files. A vector without that context cannot be re-baselined intelligently when a dependency moves, because nobody can tell whether the change was legitimate.
A justified tolerance. The tolerance is a claim about numerical behaviour. Exact equality is right while the environment is pinned; a loose tolerance chosen to make a red test pass hides exactly what the vector exists to reveal.
Complete Runnable Implementation
from __future__ import annotations
from dataclasses import dataclass
import numpy as np
from pyproj import CRS, Transformer
@dataclass(frozen=True)
class GoldenVector:
"""One independently sourced transformation case."""
name: str
source_crs: str
target_crs: str
source_xy: tuple[float, float]
expected_xy: tuple[float, float]
tolerance_m: float
provenance: str # where the expected value came from
operation_id: str | None = None # pin the operation, not just the CRS pair
def __post_init__(self) -> None:
if not self.provenance:
raise ValueError(
f"{self.name}: a vector with no provenance is a record of current "
f"behaviour, not a test"
)
if self.tolerance_m <= 0.0:
raise ValueError(f"{self.name}: tolerance must be positive")
def run(self) -> tuple[float, float]:
"""Transform the source coordinate with a pinned operation where given."""
if self.operation_id:
tf = Transformer.from_pipeline(self.operation_id)
else:
tf = Transformer.from_crs(
CRS.from_user_input(self.source_crs),
CRS.from_user_input(self.target_crs),
always_xy=True,
)
x, y = tf.transform(*self.source_xy)
return float(x), float(y)
def check(self) -> None:
got = np.array(self.run(), dtype=np.float64)
want = np.array(self.expected_xy, dtype=np.float64)
delta = np.abs(got - want)
if np.any(delta > self.tolerance_m):
raise AssertionError(
f"{self.name}: off by {delta.max() * 1000:.4f} mm "
f"(tolerance {self.tolerance_m * 1000:.4f} mm)\n"
f" source {self.source_crs} -> {self.target_crs}\n"
f" operation {self.operation_id or 'chosen by PROJ'}\n"
f" provenance {self.provenance}"
)Parameter Reference
| Field | Type | Units | Note |
|---|---|---|---|
source_xy, expected_xy |
tuple[float, float] |
degrees or metres | Axis order per always_xy=True |
tolerance_m |
float |
metres | Exact (1e-9) while the environment is pinned |
provenance |
str |
— | Required; a vector without it is not independent |
operation_id |
str or None |
— | A pipeline string pins the operation itself |
check() |
— | — | Raises with the context needed to diagnose |
Worked Example
vectors = [
GoldenVector(
name="nad83_2011_geographic_to_utm10n",
source_crs="EPSG:6318", target_crs="EPSG:6339",
source_xy=(-122.600000, 45.500000),
expected_xy=(531_247.3195, 5_037_223.0894),
tolerance_m=1e-4,
provenance="agency worked example, sheet 4, 2019 revision",
),
GoldenVector(
name="grid_boundary_west_edge",
source_crs="EPSG:4267", target_crs="EPSG:6318",
source_xy=(-124.000000, 45.000000),
expected_xy=(-124.001178, 45.000059),
tolerance_m=1e-6,
provenance="published control pair CV-114 / CV-114A",
),
]
for v in vectors:
v.check()The second vector is the more valuable of the two, and its name says why: it sits exactly on a grid boundary, where an off-by-one in the cell index, a clamped extent or a wrong sub-grid selection all produce wrong answers that a mid-grid vector will never expose.
Validation Check
def assert_vectors_are_spread(vectors: list[GoldenVector]) -> None:
"""A vector set clustered in one place tests one cell of one grid."""
xs = np.array([v.source_xy[0] for v in vectors])
ys = np.array([v.source_xy[1] for v in vectors])
assert xs.ptp() > 1.0 and ys.ptp() > 1.0, (
"golden vectors span less than a degree — add cases at the grid edges, "
"inside a nested sub-grid, and outside every grid"
)Common Mistakes
Vectors generated by the implementation. The commonest and the most understandable, because it is so easy: run the code, paste the output, commit. The result passes forever and proves nothing. If no independent source exists for an operation, say so in the provenance field — an honest “generated by this implementation on 2026-08-11, pending independent confirmation” is a note a reviewer can act on.
A tolerance loosened to make CI green. When a vector fails after a dependency upgrade, the question is which changed: the code or the environment. Answer it with the environment assertion, then re-baseline deliberately and record the version. Widening the tolerance answers nothing and disables the test permanently.
All vectors in the easy part of the domain. A set that samples the middle of one grid tests one interpolation cell. Deliberately include a point outside every grid — whose expected outcome is a refusal, not a coordinate — one on a sub-grid boundary, one in a nested child, and one at the edge of the projection zone where the series is weakest.
Choosing Which Coordinates to Pin
A useful vector set is chosen by where the code can go wrong, not by where coordinates are convenient. Six positions cover most of the failure surface for a grid-based pipeline.
Figure — six positions that cover most of the failure surface of a grid-based pipeline.
Mid-cell, mid-grid is the baseline: it should never fail, and when it does the change is global rather than a boundary case. Exactly on a node tests that interpolation is node-exact, which catches indexing and weight-ordering errors in one assertion. On a cell boundary tests that the same answer comes back whichever cell serves the query. Inside a nested sub-grid tests that the deepest grid wins rather than the first one found. Just outside every grid extent tests a refusal, and its expected outcome is an exception rather than a coordinate. At the edge of the projection zone tests the series expansion where it is weakest, which is where truncation shows up.
Six vectors per operation is a reasonable target, and their spread matters far more than their number: twenty coordinates from the same survey block test one cell of one grid twenty times, and will all pass on the day the sub-grid selection breaks.
Sourcing Vectors When No Published Pair Exists
Independent expected values are easy to find for well-travelled operations and hard for the rest, and there is a workable ladder of alternatives.
Figure — the independence ladder, best rung first.
Published control pairs are the gold standard: the same monument’s coordinates in both frames, published by the agency that computed them.
An agency worked example — the arithmetic printed in a technical bulletin — is nearly as good and often covers exactly the operation a national pipeline needs.
A second independent implementation is acceptable when the two do not share a code lineage; two wrappers around the same library are not independent, and testing one against the other proves only that the wrappers agree.
A hand computation from the published parameters is laborious and genuinely independent, and for a small parameter-based transformation it is a few lines of arithmetic worth doing once.
Self-generated, clearly labelled is the last rung, and it is legitimate as long as the provenance says so. It pins current behaviour, which catches an unintended change even though it cannot catch an existing error — and the label tells the next reader exactly how much weight to give it.
Frequently Asked Questions
Should a golden vector pin the operation or just the CRS pair?
Pin the operation wherever the result matters. A CRS pair lets PROJ choose, and its choice can change between releases or when a grid appears or disappears — which is precisely the silent change the vector is meant to detect. Keep one or two unpinned vectors as well: they test the selection logic, and their failure tells you the selection changed, which is useful information rather than noise.
What tolerance should a vector against published control carry?
The uncertainty of the published control, not the numerical precision of the arithmetic. A control coordinate published to the centimetre cannot support a millimetre assertion, and pretending otherwise produces a test that fails for reasons that have nothing to do with the code. Vectors against a second implementation, by contrast, can be held to a micrometre.
How do I re-baseline after a legitimate PROJ upgrade?
Deliberately, in its own commit, with the environment record updated in the same change. Re-run the vectors, inspect the differences — a uniform few-digit change across all of them is a series refinement, a centimetre shift in one region is a changed operation — and record which of the two you concluded. That commit message is the audit trail for why the numbers moved.
Do golden vectors replace testing against control points?
No: they answer different questions. A golden vector asks whether the software still does what it did; a control comparison asks whether what it does matches the ground. A pipeline needs both, and the control comparison is the one that appears in the deliverable, as validating datum alignment with control points describes.