Geospatial File Formats and CRS Metadata

Most coordinates that go wrong in a cadastral pipeline were transformed perfectly and then written to a file that could not carry what they meant. This topic, part of batch transformation and automation, is about the boundary between correct arithmetic and a defensible deliverable: which formats can hold a realisation, an epoch and an operation identifier, what each one silently discards, and how to check that what you meant to write is what a reader will find.

The failure is systematic rather than accidental. Formats were designed at different times against different standards, and the older ones encode assumptions that stopped being true — that a datum has one realisation, that coordinates have no epoch, that a projection is fully described by a handful of parameters. Writing modern coordinates into those containers loses information, and the loss is silent because the file is perfectly valid.

What Each Format Can Actually Carry #

Figure — what survives each container, and what has to travel beside it.

Two rows deserve a note. GeoJSON is not a gap in the table — the specification fixes the CRS as longitude and latitude on WGS84, and a GeoJSON file carrying projected national-grid coordinates is malformed however widely it is done. And the shapefile row is the one most cadastral work actually meets: WKT1 cannot express a datum realisation unambiguously and has nowhere for an epoch, so a shapefile deliverable requires a metadata sidecar to be interpretable, as recording coordinate epochs in cadastral deliverables sets out.

The Two Losses That Matter #

Realisation collapse. A WKT1 GEOGCS names a datum, not a realisation, so NAD83(2011) and NAD83(CSRS) both round-trip through it as “NAD83”. A reader then resolves that to the ensemble code, which carries up to about two metres of ambiguity — and the file gives no hint that anything was lost.

Precision truncation. Text formats truncate at whatever precision the writer chose. Six decimal places of a degree is 0.11 m; a shapefile’s coordinates are float64 internally but its attribute table is not, so a coordinate copied into an attribute column can quietly lose most of its precision. The rule from validating coordinate precision to millimetre standards applies at every format boundary: eight decimal places for degrees, four for metres.

Production Implementation: a Write-Then-Verify Wrapper #

from __future__ import annotations

from dataclasses import dataclass
from pathlib import Path

from pyproj import CRS

# Ensemble codes that must never appear in a deliverable: they name a family of
# realisations, not a realisation, and hide metres of ambiguity.
ENSEMBLE_CODES = {"EPSG:4326", "EPSG:4269", "EPSG:4258"}


@dataclass(frozen=True)
class WriteContract:
    """What a deliverable must carry, checked before and after writing."""

    crs: CRS
    epoch: float | None
    operation_id: str
    decimals_degrees: int = 8
    decimals_metres: int = 4

    def check_crs(self) -> None:
        code = self.crs.to_authority()
        if code and f"{code[0]}:{code[1]}" in ENSEMBLE_CODES:
            raise ValueError(
                f"{code[0]}:{code[1]} is a datum ensemble; write the realisation "
                f"the coordinates are actually in"
            )
        if self.crs.is_geographic and self.decimals_degrees < 8:
            raise ValueError(
                f"{self.decimals_degrees} decimal places of a degree resolves to "
                f"{10 ** -self.decimals_degrees * 111_320:.3f} m — use 8"
            )

    def verify_written(self, written_crs_wkt: str, written_epoch: float | None) -> None:
        """Read back what landed on disk, not what was passed to the writer."""
        got = CRS.from_wkt(written_crs_wkt)
        if not got.equals(self.crs):
            raise ValueError(
                "the CRS written to the file is not the CRS that was intended; "
                "the format may have downgraded WKT2 to WKT1"
            )
        if self.epoch is not None and written_epoch is None:
            raise ValueError(
                "the coordinate epoch did not survive the write — this format "
                "needs a metadata sidecar"
            )

The verify_written step is the one that catches the whole class of problems this topic is about. A writer that accepts a WKT2 definition and stores a WKT1 downgrade of it raises no error and produces a file that is subtly less specific than the data it holds.

Writing Coordinates at a Stated Precision #

Every format boundary is a rounding decision, and leaving it to the writer’s default is how a millimetre survey becomes a decimetre file. Three rules make the decision explicit.

Choose the precision from the units, not from habit. Eight decimal places for degrees (about 1.1 mm), four for metres (0.1 mm), and one more than you think you need for any intermediate file that will be read back and transformed again. The reasoning is in validating coordinate precision to millimetre standards, and it applies at each boundary independently: a pipeline with three text hand-offs rounds three times.

State the rounding mode. Round-half-to-even is the IEEE 754 default and the one that leaves no net bias over a large set; round-half-up shifts the mean by half a unit in the last place in a consistent direction. Over a cadastral dataset re-rounded at two boundaries, that bias is a real, measurable displacement.

Format, do not print. A bare str(x) on a float produces a platform-dependent repr, which is fine to read and unusable as the input to an audit hash. Format at the stated precision with an explicit quantisation step, exactly as generating audit hashes for transformation batches requires.

from decimal import Decimal, ROUND_HALF_EVEN


def format_coordinate(value: float, decimals: int) -> str:
    """One coordinate, one stated precision, one stated rounding mode."""
    quantum = Decimal(1).scaleb(-decimals)
    return str(Decimal(repr(value)).quantize(quantum, rounding=ROUND_HALF_EVEN))

Reading Someone Else’s File #

Incoming files deserve the mirror image of the same care, because the metadata they carry is a claim rather than a fact. Four questions settle whether a file can be used at all: is a CRS declared, is it a realisation rather than an ensemble, does the coordinate range make sense for that CRS, and — for a time-dependent frame — is there an epoch anywhere in the package?

The temptation with a file that fails one of these is to repair it from context. A shapefile from a known agency in a known region can be assigned the obvious realisation and the obvious epoch, and nine times in ten the guess is right. The tenth time it is a two-metre error that nobody can trace, because the assumption was never written down. Record every such repair as an explicit assumption in the audit block, with what it was based on, so that a downstream reader can disagree with it — the discipline set out for unmapped definitions in parsing WKT2:2019 CRS strings in Python.

Compliance: the Deliverable Package #

A cadastral deliverable is rarely one file. The package that actually satisfies an agency is usually four things: the data file, a metadata record carrying the CRS realisation, epoch, operation and grid checksums, a residual report, and a manifest listing all of them with their hashes. The manifest is what keeps them together — a sidecar that can be separated from its data file will be, and the epoch it carried is then lost exactly as surely as if it had never been written.

Figure — the four artefacts that together make a deliverable, bound by a manifest.

Worked Example: a Round Trip Through Three Formats #

The cheapest way to find out what a format discards is to write a known coordinate set into it, read it back, and compare — not the coordinates, which usually survive, but the metadata, which usually does not.

from pyproj import CRS

source = CRS.from_epsg(6318)                 # NAD83(2011) geographic
print(source.name, source.to_authority())
# NAD83(2011) ('EPSG', '6318')

# What a WKT1 .prj can express of it:
wkt1 = source.to_wkt(version="WKT1_GDAL")
back = CRS.from_wkt(wkt1)
print(back.name, back.to_authority())
# NAD83(2011) ('EPSG', '6318')      <- code recovered here
print("NAD83(2011)" in wkt1, "ENSEMBLE" in wkt1)
# True False

# What a reader that only pattern-matches the datum name will conclude:
print(CRS.from_string("NAD83").to_authority())
# ('EPSG', '4269')                  <- the ensemble: up to ~2 m of ambiguity

The middle result is the encouraging one and the last is the realistic one. Modern tooling often does recover the authority code from a WKT1 string, because the name is distinctive enough to match — but nothing in the format guarantees it, and a receiving system that resolves the datum by name rather than by code lands on the ensemble. The deliverable is then ambiguous at the metre level with no visible symptom, which is precisely why the sidecar carries the code explicitly rather than relying on a reader’s inference.

Precision at Each Boundary #

Figure — precision lost at a boundary by default, against the precision to write.

The last row is the one that surprises people. A geotransform stored at single precision is fine for a continental raster and badly wrong for a national-grid one: at a northing of six million metres, float32 resolution is about half a metre, so every pixel edge is displaced by an amount no viewer will show and every parcel overlay will inherit.

Failure Modes #

• A shapefile shipped without a sidecar. The .prj names a datum without a realisation, so the deliverable is ambiguous at the metre level with no indication of it.
• GeoJSON with projected coordinates. Valid JSON, invalid GeoJSON, and readers will treat the numbers as degrees.
• A CRS downgraded on write. WKT2 in, WKT1 on disk, realisation and epoch gone; only reading the file back detects it.
• Precision truncated in an attribute column. Coordinates copied into attributes for convenience and rounded to two decimals, then used downstream as though authoritative.
• Axis order flipped by the format. Some readers apply the CRS axis order and some assume longitude-latitude; writing without asserting the convention produces a file that is right in one tool and transposed in another, the problem diagnosed in handling CRS mismatches in cadastral datasets.

A Format Decision Table #

The choice of container is usually made once per pipeline and inherited for years, so it is worth making deliberately. Four questions settle it.

Does the recipient specify a format? Then that is the answer, and everything below becomes a question about what to ship alongside it. Agencies specify shapefiles more often than anything else, which is why the sidecar pattern matters so much in this field.

Does the data need to carry a coordinate epoch? If the frame is time-dependent, a container that cannot hold one is only half of a deliverable. GeoPackage with WKT2 can; a shapefile cannot; LandXML can by convention.

Will the file be read by software you do not control? The more diverse the consumers, the more the identity has to be stated redundantly — in the CRS definition, in the sidecar and in the transmittal — because each consumer reads a different subset.

Is the file an archive or a working artefact? An archival copy should be the most self-describing option available and should be hashed and signed. A working intermediate can be anything, provided it never escapes — and intermediates escape constantly, which is an argument for treating them the same way.

Answering those four in order takes a few minutes and produces a decision that survives a change of personnel, which is more than can be said for a format chosen because it was what the last project used.

Checking a Package Before It Ships #

The last thing to do before a deliverable leaves is to read it back as a stranger would: open the data file with no knowledge of how it was produced, and see whether the CRS, the realisation, the epoch and the operation can all be recovered from what is in the package. If any of the four requires knowledge that only exists in the producer’s head or project file, it is missing from the deliverable, however carefully it was computed.

That check takes a minute per package and catches the failures this topic is about — all of which look identical from the producing side, where the information is obviously present, and only become visible from the consuming side, where it is not.

Frequently Asked Questions #

Which format should a new cadastral pipeline standardise on? #

GeoPackage, unless the receiving agency specifies otherwise. It carries WKT2, so the realisation survives; it is a single file, so nothing gets separated; and it holds real float64 geometry rather than text. Where a shapefile is mandated — and it often is — produce it as an additional artefact alongside a GeoPackage or metadata sidecar rather than as the authoritative one.

Is it acceptable to ship coordinates in a CSV? #

Yes, if a companion metadata file ships with it and the manifest binds the two together. CSV has the virtue of being unambiguous about precision: you control exactly how many digits are written. What it cannot do is carry any CRS information at all, so it is only ever half of a deliverable.

How do I know whether a writer downgraded my CRS? #

Read the file back and compare the parsed CRS against the one you intended, which is what verify_written does. Comparing the WKT strings is not enough — serialisations differ harmlessly — so compare semantically, and check separately that the epoch survived, because it is the field most likely to be dropped.

Do these concerns apply to raster deliverables? #

Yes, with one addition: a raster also carries a geotransform, and its precision matters as much as the coordinates’. A geotransform written at single precision places pixel edges tens of centimetres from where they belong at national-grid magnitudes, which is invisible in a viewer and decisive in a boundary retracement.

What about formats that carry no CRS but are used anyway, like DXF? #

They are common in cadastral practice and they need the full sidecar treatment, because the format contributes nothing at all to the identity of the coordinates. Two additional points apply. CAD formats routinely carry coordinates in feet, sometimes US survey feet rather than international feet — a difference of two parts per million, or about twelve metres across a state — so the unit belongs in the sidecar explicitly. And CAD data is often in a local coordinate system tied to the national grid by a translation and a rotation that lives in the surveyor’s notes; publish that tie as a documented transformation or the drawing is not georeferenced at all, whatever the coordinates look like.

The recurring theme across every format in this topic is that a container which cannot express the realisation, the epoch and the operation does not stop being usable — it stops being self-sufficient. Recognising which of the two you are shipping, and packaging accordingly, is the whole of the discipline; everything else here is detail about particular containers.

Related #

• Batch transformation and automation — the parent section
• Reading and writing .prj files with Python
• Transforming GeoPackage layers without losing precision
• Exporting LandXML with correct CRS metadata
• Exporting ISO 19111 metadata for cadastral deliverables