A GeoTIFF Checksum Cannot Prove You Copied Its Validity Mask

A GeoTIFF arrives at a receiving directory. Its TIFF SHA-256 matches the producer’s TIFF exactly. Its 3×3 raw array is unchanged. Its shape, uint8 dtype, CRS, transform and nodata metadata are unchanged. Even the receiving reader reports eight valid cells.

Yet the dataset is wrong.

The intended valid population contains the top-left value 0 and excludes the bottom-right value 99. If an external validity-mask sidecar is omitted during copying, a mask-aware reader can instead fall back to nodata metadata: it excludes 0 and admits 99. The count remains eight, but the expected valid-pixel sum changes from 35 to 134 and the mean from 4.375 to 16.75.

That is the acceptance problem addressed here: did the receiving process reconstruct the intended valid-pixel population, including the associated mask, rather than merely reproduce unchanged TIFF bytes?

The fixture is deliberately synthetic, local and single-band. The laboratory described below has not been executed for this article; numerical outcomes are authored arithmetic expectations, while runtime observations remain pending until the protocol is run in a recorded environment.

The receiving reader is therefore the acceptance boundary. A successful copy command, a matching TIFF checksum, or even a matching valid-cell count is not enough. Acceptance requires component evidence plus a coordinate-level validity oracle.

Define the delivered raster contract before copying files

Treat the delivered dataset as the unit of acceptance, not the pathname ending in .tif.

That distinction matters because GDAL has a formal band-mask model in which validity can come from a dedicated mask rather than from the raster sample values themselves. GDAL RFC 15: Band Masks defines mask pixels as unsigned bytes where zero means invalid and nonzero normally means valid. It also documents recognition of external .msk datasets and nodata-derived fallback masks.

For this fixture, the producer contract is intentionally narrow:

The canonical contract is fixed before any copy occurs. Neither the source mask returned by read_masks() nor a candidate’s pixel values are allowed to redefine it.

That is the practical distinction between component identity, logical raster identity, and validity identity. A matching TIFF hash proves that one physical component is unchanged. Equal raw arrays and georeferencing establish another layer of equivalence. Neither proves that the same cells are considered usable by the consumer.

This article stays at that delivery boundary rather than duplicating broader remote-sensing processing and validation foundations. The question is not what the pixels scientifically represent. It is whether an already approved raster-plus-mask contract survives packaging and reception. Refonte’s broader remote-sensing material covers substantially wider raster, preprocessing and validation concerns.

Name both parties in the run record: the producer that declares the bundle and the receiving consumer that performs the acceptance read. “The TIFF looks correct” is not a dataset contract.

Pin the writer, consumer and mask-storage setting

Mask storage must be explicit because the documentation itself gives a useful warning against relying on remembered defaults.

The current Rasterio Nodata Masks tutorial says that a mask written with write_mask() is saved by default as a sidecar GeoTIFF, while the current GDAL GeoTIFF driver reference says GDAL_TIFF_INTERNAL_MASK defaults to TRUE starting with GDAL 3.9 and was FALSE in GDAL 3.8 or earlier. The correct engineering response is not to decide that one documentation page is universally “right”; it is to set the configuration explicitly, record the installed runtime, and inspect what was actually created.

Rasterio itself recommends setting GDAL configuration through rasterio.Env; its options documentation shows GDAL_TIFF_INTERNAL_MASK=True as the mechanism for requesting internal GeoTIFF masks.

Record at least:

•         the full Python version and build string;

•         NumPy and Rasterio versions;

•         the GDAL version linked to Rasterio;

•         operating system and architecture;

•         GeoTIFF driver presence;

•         fixture revision and commissioning batch;

•         explicit writer mask option;

•        consumer version and exact read operations.

This should be the same receiving read path for every candidate. Do not let the failing case use one set of options and the corrected control another.

The code below creates no remote dependency and downloads no imagery. It works only with an already provisioned Python environment containing NumPy and Rasterio. The example uses a declared CRS and transform solely to verify metadata preservation; they have no scientific interpretation.

from future import annotations

import argparse
import hashlib
import json
import os
import platform
import shutil
import sys
from pathlib import Path

import numpy as np
import rasterio
from rasterio.crs import CRS
from rasterio.transform import Affine


FIXTURE_REVISION = "geotiff-mask-fixture-r1"
COMMISSIONING_BATCH = "2026-09-25"
MARKER = ".geotiff_mask_acceptance_fixture"

DATA = np.array(
    [
        [0, 2, 3],
        [4, 5, 6],
        [7, 8, 99],
    ],
    dtype=np.uint8,
)

VALIDITY_MASK = np.array(
    [
        [255, 255, 255],
        [255, 255, 255],
        [255, 255,   0],
    ],
    dtype=np.uint8,
)

ORACLE_VALID = VALIDITY_MASK != 0

CRS_VALUE = CRS.from_epsg(4326)
TRANSFORM_VALUE = Affine(
    0.01, 0.00, -1.00,
    0.00, -0.01, 1.00,
)

PROFILE = {
    "driver": "GTiff",
    "height": 3,
    "width": 3,
    "count": 1,
    "dtype": "uint8",
    "crs": CRS_VALUE,
    "transform": TRANSFORM_VALUE,
    "nodata": 0,
}

CONSUMER_OPTIONS = {
    "band": 1,
    "window": None,
    "out_shape": None,
    "resampling": None,
    "sharing": False,
    "operations": [
        "read(1)",
        "read_masks(1)",
        "read(1, masked=True)",
    ],
}


def sha256(path: Path) -> str:
    h = hashlib.sha256()
    with path.open("rb") as f:
        for chunk in iter(lambda: f.read(1024 1024), b""):
            h.update(chunk)
    return h.hexdigest()


def safe_remove_run(root: Path) -> None:
    root = root.resolve()
    marker = root / MARKER
    if not root.exists():
        return
    if not marker.is_file():
        raise RuntimeError(
            f"Refusing to delete unmarked directory: {root}"
        )
    shutil.rmtree(root)


def coords(valid: np.ndarray) -> list[list[int]]:
    return [[int(r), int(c)] for r, c in np.argwhere(valid)]


def inspect_dataset(path: Path) -> dict:
    with rasterio.open(path, "r", sharing=False) as ds:
        raw = ds.read(1)
        gdal_mask = ds.read_masks(1)
        masked = ds.read(1, masked=True)

        gdal_valid = gdal_mask != 0
        numpy_valid = ~np.ma.getmaskarray(masked)

        if not np.array_equal(gdal_valid, numpy_valid):
            raise AssertionError(
                "GDAL-valid and NumPy-masked representations disagree"
            )

        values = raw[gdal_valid]

        return {
            "path": str(path.resolve()),
            "files": [str(Path(p).resolve()) for p in ds.files],
            "raw": raw.tolist(),
            "shape": list(raw.shape),
            "dtype": str(raw.dtype),
            "crs": ds.crs.to_string() if ds.crs else None,
            "transform": list(ds.transform),
            "nodata": ds.nodata,
            "mask": gdal_mask.tolist(),
            "mask_flags": [
                [flag.name for flag in band_flags]
                for band_flags in ds.mask_flag_enums
            ],
            "valid_coordinates_rc": coords(gdal_valid),
            "valid_count": int(gdal_valid.sum()),
            "valid_sum": int(values.astype(np.int64).sum()),
            "valid_mean": float(values.mean()),
            "oracle_mask_equal": bool(
                np.array_equal(gdal_valid, ORACLE_VALID)
            ),
            "raw_equal": bool(np.array_equal(raw, DATA)),
        }


def manifest(directory: Path) -> list[dict]:
    return [
        {
            "name": p.name,
            "bytes": p.stat().st_size,
            "sha256": sha256(p),
        }
        for p in sorted(directory.iterdir())
        if p.is_file() and p.name != MARKER
    ]


def environment_manifest() -> dict:
    with rasterio.Env() as env:
        gtiff_description = env.drivers().get("GTiff")

    gdal_version = getattr(rasterio, "__gdal_version__", None)
    if not gdal_version:
        raise RuntimeError(
            "Could not determine GDAL version linked to Rasterio"
        )

    return {
        "fixture_revision": FIXTURE_REVISION,
        "commissioning_batch": COMMISSIONING_BATCH,
        "python": sys.version,
        "python_executable": sys.executable,
        "python_build": platform.python_build(),
        "python_implementation": platform.python_implementation(),
        "numpy": np.__version__,
        "rasterio": rasterio.__version__,
        "gdal_linked_by_rasterio": gdal_version,
        "operating_system": platform.platform(),
        "machine": platform.machine(),
        "gtiff_driver_description": gtiff_description,
        "preexisting_shell_GDAL_TIFF_INTERNAL_MASK":
            os.environ.get("GDAL_TIFF_INTERNAL_MASK"),
        "writer_external_mask_config": {
            "GDAL_TIFF_INTERNAL_MASK": False
        },
        "writer_internal_mask_config": {
            "GDAL_TIFF_INTERNAL_MASK": True
        },
        "consumer_options": CONSUMER_OPTIONS,
        "profile": {
            *PROFILE,
            "crs": CRS_VALUE.to_string(),
            "transform": list(TRANSFORM_VALUE),
        },
    }


def write_external_source(path: Path) -> None:
    with rasterio.Env(GDAL_TIFF_INTERNAL_MASK=False):
        with rasterio.open(path, "w", PROFILE) as dst:
            dst.write(DATA, 1)
            dst.write_mask(VALIDITY_MASK)


def write_internal_source(path: Path) -> None:
    with rasterio.Env(GDAL_TIFF_INTERNAL_MASK=True):
        with rasterio.open(path, "w", PROFILE) as dst:
            dst.write(DATA, 1)
            dst.write_mask(VALIDITY_MASK)


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--root",
        type=Path,
        default=Path("geotiff_mask_acceptance_run"),
    )
    parser.add_argument("--reset", action="store_true")
    parser.add_argument("--cleanup", action="store_true")
    args = parser.parse_args()

    root = args.root.resolve()

    if args.cleanup:
        safe_remove_run(root)
        return

    if root.exists():
        if not args.reset:
            raise FileExistsError(
                f"{root} already exists; use --reset only for "
                "a directory created by this fixture."
            )
        safe_remove_run(root)

    root.mkdir(parents=False)
    (root / MARKER).write_text(
        f"{FIXTURE_REVISION}\n", encoding="utf-8"
    )

    dirs = {
        "source_external": root / "source_external",
        "candidate_tiff_only": root / "candidate_tiff_only",
        "candidate_full_bundle": root / "candidate_full_bundle",
        "source_internal": root / "source_internal",
        "candidate_internal": root / "candidate_internal",
    }
    for directory in dirs.values():
        directory.mkdir()

    external_tif = dirs["source_external"] / "fixture.tif"
    external_msk = dirs["source_external"] / "fixture.tif.msk"

    write_external_source(external_tif)

    # Writer is closed before these checks.
    if not external_msk.is_file():
        raise AssertionError(
            "Expected external fixture.tif.msk was not created"
        )

    source_entries = {
        p.name for p in dirs["source_external"].iterdir()
        if p.is_file()
    }
    if source_entries != {"fixture.tif", "fixture.tif.msk"}:
        raise AssertionError(
            f"Unexpected external-source inventory: {source_entries}"
        )

    source_read = inspect_dataset(external_tif)
    if not source_read["oracle_mask_equal"]:
        raise AssertionError(
            "Fresh external source read does not match validity oracle"
        )

    # Failing copy: TIFF only.
    tiff_only_path = dirs["candidate_tiff_only"] / "fixture.tif"
    shutil.copy2(external_tif, tiff_only_path)

    if set(p.name for p in dirs["candidate_tiff_only"].iterdir()) != {
        "fixture.tif"
    }:
        raise AssertionError("TIFF-only directory is not TIFF-only")

    # Passing external-bundle control.
    bundle_tif = dirs["candidate_full_bundle"] / "fixture.tif"
    bundle_msk = dirs["candidate_full_bundle"] / "fixture.tif.msk"
    shutil.copy2(external_tif, bundle_tif)
    shutil.copy2(external_msk, bundle_msk)

    # Internal-mask comparison written from authoritative mask.
    internal_source_tif = dirs["source_internal"] / "fixture.tif"
    write_internal_source(internal_source_tif)

    if (dirs["source_internal"] / "fixture.tif.msk").exists():
        raise AssertionError(
            "Internal-mask source unexpectedly has .tif.msk sidecar"
        )

    internal_candidate_tif = (
        dirs["candidate_internal"] / "fixture.tif"
    )
    shutil.copy2(internal_source_tif, internal_candidate_tif)

    evidence = {
        "environment": environment_manifest(),
        "authored_oracle": {
            "valid_coordinates_rc": coords(ORACLE_VALID),
            "valid_count": 8,
            "valid_sum": 35,
            "valid_mean": 4.375,
            "nodata_fallback_expected_coordinates_rc":
                coords(DATA != 0),
            "nodata_fallback_expected_count": 8,
            "nodata_fallback_expected_sum": 134,
            "nodata_fallback_expected_mean": 16.75,
        },
        "manifests": {
            name: manifest(path)
            for name, path in dirs.items()
        },
        "reads": {
            "source_external": source_read,
            "candidate_tiff_only":
                inspect_dataset(tiff_only_path),
            "candidate_full_bundle":
                inspect_dataset(bundle_tif),
            "source_internal":
                inspect_dataset(internal_source_tif),
            "candidate_internal":
                inspect_dataset(internal_candidate_tif),
        },
    }

    (root / "evidence.json").write_text(
        json.dumps(evidence, indent=2),
        encoding="utf-8",
    )

    print(root / "evidence.json")


if name == "__main__":
    main()

Run it in the already approved environment:

python acceptance_lab.py \
  --root ./geotiff_mask_acceptance_run \
  --reset

The resulting runtime manifest, not this article, supplies the exact build record. This separation is important: Earth-observation preprocessing context may explain the wider processing discipline, but it cannot stand in for the installed GDAL/Rasterio state used by a particular delivery acceptance run.

Create a valid zero and an invalid nonzero cell

The fixture is designed to defeat two common shortcuts at once.

The canonical raw array is:

row\col   0   1   2
0         0   2   3
1         4   5   6
2         7   8  99

Its nodata metadata value is 0, but the authoritative explicit validity mask is:

row\col    0    1    2
0        255  255  255
1        255  255  255
2        255  255    0

Under the GDAL convention, zero in that mask means invalid and nonzero means valid. That convention is explicitly documented by RFC 15 and by Rasterio’s mask tutorial.

The top-left sample at (row=0, column=0) is therefore valid despite having the same numeric value as the nodata metadata. The bottom-right 99 at (2,2) is invalid despite being nonzero.

That construction is not an obscure edge case added for drama. It directly demonstrates why valid-pixel identity cannot safely be reconstructed by guessing from the samples. Rasterio’s own tutorial notes that explicit masks can make zero-valued pixels valid despite nodata metadata.

The fixed geometry for every candidate is:

CRS.from_epsg(4326)

Affine(
    0.01, 0.00, -1.00,
    0.00, -0.01, 1.00,
)

Nothing is reprojected or resampled. The CRS and transform are delivery invariants only.

Specify the validity oracle independently

Use zero-based (row, column) coordinates. The authoritative valid set is:

(0,0) (0,1) (0,2)
(1,0) (1,1) (1,2)
(2,0) (2,1)

The corresponding sample values are:

0, 2, 3, 4, 5, 6, 7, 8

Therefore the authored arithmetic expectation is:

count = 8
sum   = 0 + 2 + 3 + 4 + 5 + 6 + 7 + 8 = 35
mean  = 35 / 8 = 4.375

Those values are an oracle, not measurements from a completed experiment. A correct execution should compare its destination mask against these coordinates. It must not first read a candidate’s mask and then define the “expected” population from whatever the candidate returned.

That separation is fundamental to reproducible acceptance: expected state must have an origin independent of the implementation under test.

Design the false reassurance of an unchanged count

Now remove the explicit mask and apply only the nodata rule sample != 0.

The expected valid coordinates become:

      (0,1) (0,2)
(1,0) (1,1) (1,2)
(2,0) (2,1) (2,2)

This set still contains eight cells. But it removed (0,0) and admitted (2,2).

Its values are:

2, 3, 4, 5, 6, 7, 8, 99

so:

count = 8
sum   = 134
mean  = 16.75

A valid-count-only acceptance rule therefore returns the same cardinality for two different populations.

The same principle applies more generally to summary statistics. This fixture makes the mean visibly different because that is useful diagnostically, but another fixture could be constructed in which two different coordinate populations have the same count and even the same mean. Exact membership is the stronger test. Statistics explain the consequence; they do not establish identity.

Write and verify an external-mask source bundle

The source fixture must itself pass a pre-copy gate.

Write the GeoTIFF under:

with rasterio.Env(GDAL_TIFF_INTERNAL_MASK=False):

and call:

dst.write_mask(VALIDITY_MASK)

The explicit False is mandatory for this scenario. Do not depend on the ambient GDAL_TIFF_INTERNAL_MASK value or on an assumed Rasterio/GDAL default.

GDAL’s GeoTIFF documentation states that the configuration controls whether a created mask is stored internally, while RFC 15 describes the .msk external-mask mechanism and its basename relationship to the principal raster.

Then close the writer. Only after closure should the acceptance harness inspect and copy the files.

For this synthetic fixture the source-directory precondition is deliberately strict:

source_external/
    fixture.tif
    fixture.tif.msk

Hash each component separately. The TIFF checksum and mask checksum serve different purposes. A later statement that “the TIFF hash matches” must never be allowed to imply that the mask also arrived.

Open source_external/fixture.tif again from disk using the receiving read path. Verify:

np.array_equal(src.read(1), DATA)
np.array_equal(src.read_masks(1) != 0, ORACLE_VALID)

The source should also expose the intended coordinate population, count 8, sum 35 and mean 4.375. Those are expected acceptance outcomes pending execution.

Rasterio documents that when a .msk exists it uses that mask instead of deriving validity from nodata metadata.

Inspect dataset.files as supporting evidence. The Rasterio I/O API defines files as the sequence of files associated with a dataset. For this controlled fixture that inventory should help confirm the two-component package.

Do not generalize the API beyond its contract. dataset.files is valuable evidence about what the driver associates with this dataset; it is not proof that every external provenance record or arbitrary publisher-specific dependency anywhere in a delivery ecosystem has been discovered.

If fixture.tif.msk was not created, or if a fresh source read fails the independent mask oracle, stop. There is no valid source control from which to judge copying.

Copy only the TIFF and inspect the receiving mask

The failing candidate must be constructed as a real copy boundary, not simulated by continuing to read an already open source handle.

Start from an empty directory:

candidate_tiff_only/

Copy only the closed source TIFF:

shutil.copy2(
    source_external / "fixture.tif",
    candidate_tiff_only / "fixture.tif",
)

Verify that the destination contains exactly one fixture component:

candidate_tiff_only/
    fixture.tif

Then reopen candidate_tiff_only/fixture.tif from that actual path.

This is where the expected failure becomes informative. RFC 15 specifies a fallback model in which a corresponding .msk is used when present; otherwise a band with nodata metadata can have a nodata-generated mask. With nodata=0 and no explicit mask available, this fixture therefore expects a Rasterio/GDAL receiving read to mark the zero invalid and nonzero samples, including 99, valid.

Because this laboratory has not been executed for the article, report that as an expected result to test, not an observed transcript.

Capture all of the following from the destination: read(1), read_masks(1), mask_flag_enums, dataset.files, nodata, dtype, CRS, transform and masked statistics. If runtime behavior differs from the documented expectation, that difference is itself a HOLD condition until explained from the recorded version and options.

Separate component identity from dataset identity

The TIFF-only control is supposed to preserve the TIFF perfectly.

Its expected component evidence is:

SHA256(source_external/fixture.tif)
    ==
SHA256(candidate_tiff_only/fixture.tif)

Its expected logical raw-raster evidence is also equal:

raw array     equal
shape         equal
dtype         equal
CRS           equal
transform     equal
nodata        equal

None of those comparisons should be deliberately broken to make the example easier.

The missing .msk does not mean the TIFF was corrupted. The correct diagnosis is narrower: the TIFF component is intact, while the declared dataset bundle is incomplete and the receiving validity population is therefore wrong.

That distinction matters operationally. Calling this “checksum failure” would send remediation toward storage corruption or transfer integrity when the actual recovery action is to restore the missing required component or deliberately repackage the authoritative mask into an accepted alternative representation.

Use the correct validity convention in every assertion

Mask inversion mistakes can invalidate the validator itself.

In GDAL/Rasterio’s validity representation:

gdal_mask = ds.read_masks(1)
gdal_valid = gdal_mask != 0

True now means valid.

By contrast:

masked = ds.read(1, masked=True)
numpy_invalid = np.ma.getmaskarray(masked)

in a NumPy masked array, True means the corresponding sample is masked, hence invalid. Rasterio explicitly documents this opposite convention and provides the relationship between a masked read and read_masks().

Convert both to the same meaning before comparison:

numpy_valid = ~np.ma.getmaskarray(masked)

assert np.array_equal(
    numpy_valid,
    ds.read_masks(1) != 0,
)

Then compare that common boolean-valid representation to ORACLE_VALID.

For the TIFF-only candidate, the authored expectation is:

oracle valid at (0,0):       True
fallback valid at (0,0):     False

oracle valid at (2,2):       False
fallback valid at (2,2):     True

Those two coordinate differences explain the entire aggregate shift from 35 to 134 while leaving the count at eight.

Deliver the complete external-mask bundle as a control

The first passing comparison is deliberately conservative: copy the source package without changing its representation.

Start with another empty destination and copy both declared components under their exact relative names:

candidate_full_bundle/
    fixture.tif
    fixture.tif.msk

Preserve a component manifest such as:

{
  "fixture_revision": "geotiff-mask-fixture-r1",
  "required_components": [
    {
      "relative_path": "fixture.tif",
      "sha256": "<recorded-at-execution>"
    },
    {
      "relative_path": "fixture.tif.msk",
      "sha256": "<recorded-at-execution>"
    }
  ]
}

The hashes are deliberately placeholders here because no laboratory run has produced them. In the actual run, they must be calculated from the written source and received files.

Close the source before the copy. Reopen only:

candidate_full_bundle/fixture.tif

for destination acceptance.

Expected passing evidence is strict: both component hashes match their source counterparts; raw samples and geometry match; read_masks(1) matches ORACLE_VALID; coordinate membership matches all eight approved cells; count is 8; sum is 35; mean is 4.375.

The mask filename is part of the packaging contract. RFC 15 says an external .msk recognized through its default mechanism must correspond to the main dataset name with .msk appended.

Therefore this is not equivalent:

candidate_wrong_name/
    fixture.tif
    copied-mask.tif.msk

Even if copied-mask.tif.msk has byte-for-byte identical content to the authoritative mask, it is a separate failing candidate unless the receiving mechanism is explicitly configured to use it. Do not award acceptance merely because “the mask file is somewhere in the folder.”

Test an explicitly internal-mask alternative

External masks are not the only acceptable representation for this test. The second comparison deliberately changes packaging while preserving the validity contract.

Write a new GeoTIFF under:

with rasterio.Env(GDAL_TIFF_INTERNAL_MASK=True):

using the same DATA, CRS, transform, nodata metadata and independently approved VALIDITY_MASK.

GDAL’s GeoTIFF reference documents the internal-mask configuration and notes that TRUE is the driver default starting with GDAL 3.9, whereas earlier GDAL releases defaulted differently. Explicitly setting True makes that historical default change irrelevant to the experiment.

After the writer closes, inspect the source directory. For this controlled comparison the expected inventory is:

source_internal/
    fixture.tif

with no external fixture.tif.msk.

Next copy only that TIFF to a fresh:

candidate_internal/

and reopen the destination.

Do not require its TIFF SHA-256 to equal the external-mask source TIFF. The file has been separately encoded with different mask packaging, so physical bytes may differ.

Instead, require logical identity:

raw array       canonical
shape           3×3
dtype           uint8
CRS             declared CRS
transform       declared transform
nodata          0
valid mask      exact oracle
coordinates     exact eight-member oracle
sum             35
mean            4.375

This comparison demonstrates an important acceptance principle: byte identity can be required within a copy scenario, while semantic equivalence can be the correct criterion for a deliberately repackaged representation.

If the supposedly internal-mask fixture unexpectedly retains an external component on which the receiving read depends, hold it. “Internal” is an observed packaging property to verify, not a label to trust.

Reconcile components, mask selection and cell membership

The acceptance record should make every dimension visible instead of reducing the run to one green boolean.

The following table shows the expected state under the documented model. Actual execution must replace “expected” with recorded evidence before any production decision is signed.

This is the point at which packaging quality connects to downstream satellite-analysis workflows: downstream processing only receives the population exposed by its actual read path. The broader Refonte article discusses AI and satellite-analysis applications, but it does not substitute for component-level raster acceptance.

A useful run-specific evidence record should retain, for every candidate:

component names
component sizes
per-component SHA-256
actual destination path
raw-array equality
shape and dtype
CRS and transform
nodata metadata
mask flags
returned GDAL-style mask
boolean-valid coordinates
oracle equality
valid count
valid sum
valid mean
consumer options
runtime versions
decision

An error in one required field remains visible even when everything else passes.

Explain which mask the consumer actually selected

Do not simply record the resulting mask; record why the reader appears to have selected it.

RFC 15 defines mask flags including GMF_PER_DATASET, GMF_NODATA, GMF_ALPHA and GMF_ALL_VALID. It also documents default mask behavior in which a corresponding .msk has priority over a nodata-generated mask, followed by other fallbacks.

Rasterio exposes mask information through read_masks() and mask-flag metadata, while its tutorial specifically says an existing .msk causes Rasterio to use that mask instead of nodata metadata.

For the external source and full-bundle control, the expected evidence is therefore an explicit per-dataset mask rather than nodata-derived validity.

For the TIFF-only candidate, GMF_NODATA-style evidence is expected because nodata=0 remains while the .msk has disappeared.

For the internal-mask comparison, an explicit dataset mask is expected inside the TIFF.

Treat those as expectations to reconcile with actual mask_flag_enums, not strings to hard-code into a success transcript.

The scope is also consumer-specific. This article accepts a Rasterio/GDAL receiving path. It does not claim that an unrelated image viewer, analytics engine or proprietary application will honor the same mask semantics automatically.

Make membership checks stronger than summary checks

The decisive comparison is:

np.array_equal(candidate_valid, ORACLE_VALID)

and, for an auditable human-readable form:

candidate_coordinates == approved_coordinates

The count of eight is then a derived diagnostic.

For the failing copy, a coordinate set difference should show:

missing from received valid population:
    (0,0)

unexpectedly added:
    (2,2)

That is much stronger evidence than:

valid_count == 8

Statistics still matter. A jump in mean from the oracle’s expected 4.375 to the nodata fallback’s expected 16.75 makes the consequence intuitive. But statistics remain secondary because they can collide. Two masks can select different cells while producing the same count, sum, mean, minimum or maximum.

The acceptance oracle therefore proceeds from exact cell membership outward to summaries, not the other way around.

Resolve documentation defaults through explicit configuration

The apparent documentation tension around mask storage is worth preserving because it is exactly the kind of detail that causes reproducibility failures when engineers depend on memory.

The stable Rasterio mask tutorial currently says that a mask created with write_mask() is saved to a sidecar by default. Rasterio’s options page likewise illustrates rasterio.Env(GDAL_TIFF_INTERNAL_MASK=True) for putting masks inside GeoTIFFs and describes the option as GDAL configuration.

The current GDAL GeoTIFF driver documentation, meanwhile, says:

GDAL_TIFF_INTERNAL_MASK = TRUE

is the default from GDAL 3.9 onward, with FALSE as the default in GDAL 3.8 or earlier.

Those statements should not be flattened into “Rasterio always creates external masks” or “GeoTIFF masks are always internal now.”

There are at least three distinct facts to record:

GDAL 3.9 belongs in the article as historical version context for the default change. It is not a 2026 feature announcement.

The engineering control is therefore simple: force False for the external-mask source; force True for the internal-mask comparison; close the datasets; inspect the directories; then reopen from the actual destination paths.

That procedure makes the test resilient to default changes without pretending defaults do not matter. Defaults still matter in uncontrolled pipelines, which is precisely why acceptance evidence should expose them.

A future substantive Rasterio or GDAL change is a revalidation trigger. Do not silently run a different stack and paste its output into the same acceptance record. Create a new run manifest showing the changed build, options and fixture revision.

Recover only from an approved validity source

Once the TIFF-only candidate fails the validity oracle, quarantine it as an incomplete dataset package. Do not “repair” it by searching its sample values for zeros.

Three recovery paths are legitimate.

First, when the approved external source bundle still exists, recopy the bundle. Copy both fixture.tif and fixture.tif.msk, preserve their relative naming, verify both hashes and perform a new destination read. This is the preferred recovery for an accidental component omission because it does not alter representation.

Second, when delivery policy requires a one-file artifact, repackage from the approved explicit mask. Write a separate GeoTIFF with GDAL_TIFF_INTERNAL_MASK=True, then validate its raw pixels, geometry and exact mask at the receiving path. The resulting TIFF need not share the original external-source TIFF hash.

Third, when no authoritative mask remains, hold.

The third case is the most important discipline in the playbook. Nodata metadata does not recover the missing validity evidence for this fixture: it produces the wrong membership by construction. A plausible value-based rule is not provenance.

Never apply a generic repair such as:

valid = data != 0

because (0,0) is deliberately valid. Likewise, do not declare 99 invalid merely because its magnitude makes it look suspicious. Scientific interpretation of sample values is outside this delivery test.

Recovery therefore follows authority:

approved external mask available
    -> recopy

approved mask available but packaging must change
    -> repackage and revalidate

approved mask unavailable
    -> hold

The absence of evidence is not permission to manufacture the missing mask.

Revalidate the downstream consumer before reuse

A complete bundle and a correct GeoTIFF mask are necessary evidence for this acceptance path, but they do not force every downstream application to honor validity.

Take the candidate that passed packaging acceptance and execute the actual in-scope consuming read against it. Record the application or library version, open options, band selection, masked-read mode and any operation that could change mask interpretation.

For the Rasterio acceptance path, keep the read explicit:

with rasterio.open(candidate, "r", sharing=False) as ds:
    raw = ds.read(1)
    mask = ds.read_masks(1)
    masked = ds.read(1, masked=True)

Then require both:

np.array_equal(mask != 0, ORACLE_VALID)

and:

np.array_equal(
    ~np.ma.getmaskarray(masked),
    ORACLE_VALID,
)

Rasterio documents both the GDAL-style validity mask and the inverse NumPy masked-array convention, making this a suitable consumer-level reconciliation for the declared path.

A consumer that ignores explicit masks can still read all nine raw sample values correctly. That does not make it an accepted consumer for a workflow whose dataset contract includes validity.

This distinction keeps packaging validation separate from the separate task of model evaluation. A raster-delivery test answers whether the downstream operation received the declared inputs and mask semantics. It does not establish the accuracy of a classifier, the quality of scientific interpretation, or the fitness of a later model. The linked Refonte article addresses model-validation concerns at a different layer.

Similarly, a correct validity population does not validate radiometry, cloud classification, calibration or any Earth-observation inference. Those topics are intentionally outside this fixture.

Preserve the consumer check in the acceptance artifact. Otherwise a future team can possess the right files while silently using a read path that does not consume the mask contract.

Decide accept, recopy, repackage or hold

The decision should follow evidence, not intuition about what “probably happened.”

“RECOPY” and “REPACKAGE” are actions, not retroactive declarations that the failed candidate was acceptable.

An external bundle that passes this Rasterio/GDAL consumer test can be accepted for the declared pairing:

fixture revision
+ component manifest
+ runtime
+ receiving read procedure

Do not inflate that decision into “the raster is valid for every consumer.”

Likewise, ACCEPT should require every contract dimension simultaneously:

required components satisfied
AND raw array equal
AND shape equal
AND dtype equal
AND CRS equal
AND transform equal
AND nodata metadata equal
AND validity mask equal
AND coordinate population equal
AND expected statistics reconciled
AND consumer path recorded

For the external full-bundle copy, component hash identity is appropriate. For the internal repackaging comparison, requiring byte identity with the original external TIFF would be the wrong criterion because representation has intentionally changed.

That is why the component manifest and validity oracle must coexist. One records what physical things should be delivered; the other records the logical population the consumer must reconstruct.

Assign producer and consumer ownership of the bundle

Packaging defects persist when ownership stops at “I generated the TIFF.”

The producing side owns the authoritative validity source and the delivery declaration. For this fixture that means defining the raw raster, explicit mask, fixture revision, required filenames and per-component hashes. It also owns the writer configuration that determines whether the validity mask is external or internal.

The transfer function owns preservation of the declared bundle. It should not decide independently that .tif.msk “looks optional” because the main raster opens without it.

The receiving side owns verification from destination storage. It inventories the delivered components, recalculates hashes, opens the destination path, reads the mask and compares exact coordinate membership against the oracle.

The acceptance reviewer owns the decision record. A reasonable immutable run artifact contains:

batch identifier
fixture revision
source manifest
destination manifest
all component hashes
writer configuration
Python/NumPy/Rasterio/GDAL versions
OS information
consumer options
mask flags
coordinate-level comparison
summary statistics
decision and reason
timestamp/run identifier

Rasterio’s dataset.files can supplement that record because the API exposes associated files, but the publisher’s component manifest remains the governing statement of what this known fixture requires.

That division of responsibility complements the broader discussion of remote-sensing and data-analysis responsibilities, where imagery preparation, analysis and decision support occupy related but distinct responsibilities. Here, the boundary is much narrower: producer intent and consumer reconstruction must be reconciled through evidence.

Trigger revalidation when the GeoTIFF driver changes materially, Rasterio or GDAL changes, mask-storage configuration changes, naming rules change, the component manifest changes, a different consumer is introduced, or an external mask is repackaged internally.

A filename such as fixture.tif is not a versioned delivery record. The immutable manifest and fixture revision are.

Build the remote-sensing foundations behind reliable delivery

This small fixture exercises a broader professional habit: separate raw measurements from validity evidence, make preprocessing state reproducible, and verify what a downstream reader actually consumes.

Refonte Learning’s published Remote Sensing Scientist/Engineer curriculum lists Python-based Earth-observation work, preprocessing and quality masking among its foundations, alongside change detection and time-series analysis. The page currently describes a three-month program at roughly 10–12 hours per week and lists a Training Certificate and Certificate of Internship on successful completion. It does not establish that this particular Rasterio/GDAL .msk acceptance laboratory is taught, so this exercise should not be presented as a named curriculum module.

For engineers strengthening those foundations, the published curriculum is worth inspecting as one structured learning route.

The operational standard remains independent of enrollment: accept the raster only when the receiving consumer reconstructs the declared eight-cell population, specifically (0,0) through (2,1) except (2,2), from traceable components and recorded configuration. A matching TIFF checksum proves that the TIFF survived. The manifest and coordinate-level mask oracle prove whether the intended dataset did.