Build engineers reviewing a partial GNU Make target after a failed recipe

Why GNU Make Can Skip a Target Left by a Failed Recipe

Thu, Oct 29, 2026

A generator fails loudly, GNU make returns nonzero, and the file at the target pathname is visibly incomplete. Then someone fixes the generator, runs make again, sees exit status 0, and assumes the artifact was repaired. That second green can be false. The surviving partial file may already be newer than every declared prerequisite, so the next invocation never executes the producer at all. The GNU make manual on recipe errors documents this exact hazard: after an unsuccessful recipe changes a target, the target's timestamp can make a later invocation consider it current unless cleanup removes it.

This article follows one ordinary text target across a failed invocation and a separate retry. The supplied preparation run used GNU Make 4.3, CPython 3.12.14, Linux 6.18.55, glibc 2.39, and /bin/sh resolving to dash. The fixture is owned, disposable, serial, and synthetic; the observations below come from that supplied preparation run. The operational question is narrow: which bytes remain after failure, and will a later invocation actually regenerate them?

The answer requires three kinds of evidence at once: target bytes, target timestamp, and producer-invocation count. A successful content check still does not authorize software release; it only proves the bounded target contract used here.

Define the recovery contract and freshness boundary

Before discussing cleanup, define what the target is supposed to mean. The requested new artifact is exactly BEGIN\nrelease=v2\nEND\n. Call that GOOD. The previously valid artifact is BEGIN\nrelease=v1\nEND\n; call it OLD. The deliberately incomplete write is BEGIN\nrelease=v2\n, missing the closing END\n; call it PARTIAL. These are byte contracts, not labels assigned after looking at outcomes.

That distinction matters because four filesystem states have different operational meanings. An absent target can force rebuilding under the ordinary file rule. A PARTIAL target exists but is invalid. OLD is structurally valid but does not satisfy the requested v2 build. GOOD satisfies this fixture's content contract. GNU make's freshness decision is narrower: for this ordinary rule, a missing target or a normal prerequisite newer than the target makes it out of date. That rule says nothing about whether the file's application content is complete.

Three independent decisions follow. First, content acceptance asks whether the final bytes equal GOOD. Second, execution evidence asks whether the generator actually ran on the invocation being interpreted. Third, release approval belongs to a downstream authority and cannot be inferred from either a zero exit code or a matching synthetic file.

The generated file also needs an owner. In this fixture, the recipe and its helper own artifact.txt, the same-directory candidate, and the disposable case directory. That ownership makes an owned retry meaningful: the operator can rerun a generator whose side effects are known and repeatable. It does not justify rerunning arbitrary release automation.

Use four decisions consistently. HOLD means the available evidence cannot support use of the target. REPAIR means a Makefile or publication-policy defect has been identified and should be corrected before relying on another run. Owned retry means rerun only after the state that caused the false result has been restored to a safe boundary. ACCEPT means only that this bounded target contract passed; it is not a release authorization.

Separate freshness checks from file validity

The fixture uses one explicit target and two fixed normal prerequisites:

artifact.txt: input.txt builder.py
   "$$FIXTURE_PYTHON" builder.py --build

The GNU make rule example states the ordinary rule semantics plainly: the target is out of date if it does not exist, or if either normal prerequisite is newer. The recipe line is identified by a TAB. In the fixture, input.txt is deliberately newer than an initial OLD target, while both prerequisites remain fixed during each case. Once a failing direct write gives artifact.txt a newer timestamp, make can later conclude that no recipe is required even though the bytes are PARTIAL.

This is a different fault from an incomplete dependency graph. The Refonte Learning article on how Make tracks header dependencies examines inputs that Make was never told to track. Here, the graph is intentionally complete and unchanged: input.txt and builder.py are both listed, the target recipe does run on the failing attempt, and the recovery problem begins after that recipe has modified its output.

Freshness therefore cannot substitute for validity. Modification time is evidence about ordering, not a checksum, schema validator, framing check, or release policy. The preparation ledger uses content snapshots precisely because a current-looking timestamp can coexist with incomplete bytes. GNU make itself describes the hazard: a recipe can fail after changing the target, yet the resulting timestamp may cause the next invocation not to try again.

Keep every prerequisite fixed

input.txt and builder.py are the only normal prerequisites, and the explicit goal is always artifact.txt. Their timestamps are assigned deterministically: builder.py is at T0, input.txt is two seconds newer, and an initial OLD target is one second newer than builder.py but one second older than input.txt. No sleep is needed.

BUILD_MODE is only controlled fault injection. It selects whether the helper writes GOOD, PARTIAL, stages a candidate, or restores an mtime. It is deliberately not declared as a Make dependency. M02 is valuable because changing BUILD_MODE from fail to good does not force rebuilding; the unchanged attempt log proves that make skipped the producer. That is the intended control, not a recommendation to hide real build inputs in environment variables.

Identify the recipe and cleanup policy

The reference preparation run identified /usr/bin/make as GNU Make 4.3, recorded its executable hash, set SHELL := /bin/sh, set .SHELLFLAGS := -c, and reported /bin/sh resolving to /usr/bin/dash. Every make invocation uses --no-print-directory -rR -j1 -f Makefile artifact.txt: no built-in rules or variables, one job, one explicit makefile, one explicit file goal. The controller supplies a small environment and an explicit working directory to subprocess.run; the Python 3.12 subprocess documentation states that run() waits for completion and can control cwd, env, timeout, output capture, and the returned process status.

The generator's deliberate failure codes and make's own process status are separate layers. The helper returns 7 for several producer failures and 65 for candidate validation failure, while the observed GNU make process returns 2 for the unignored recipe failures in this fixture. The test asserts the make status actually observed rather than assuming it will numerically mirror the child code. GNU make's documented rule is semantic: after a shell invocation returns nonzero, make treats that recipe line as an error unless configured to ignore it.

The Makefile text is generated inside each case directory, and the recipe begins with a real TAB, not spaces. The fixture does not exercise changing directories inside recipes or .ONESHELL; the working directory is supplied from the controller and kept fixed. For that separate concern, see recipe shell context in Makefiles.

Check what make was told to ignore

Before interpreting a failed command, inspect the policy declarations. A leading - tells GNU make to ignore that recipe line's error; GNU make then treats the error like success for continuation purposes while reporting that it was ignored. .IGNORE can broaden that behavior, while the manual's special built-in targets define .DELETE_ON_ERROR as deleting a changed target when its recipe exits nonzero.

The fixture executes only the selective leading-minus control in M06. It does not test -i, global .IGNORE, .ONESHELL, pipeline wrappers, or shell-option propagation. Those are documented context, not hidden variables in this laboratory.

Run the owned twelve-case fixture

Save the following block exactly as cloud_make_failure_fixture.py, including the final newline. Run it with Python 3.12+ on a POSIX system where GNU make is available. The preparation source was 699 whitespace-separated words and 174 lines, with SHA-256 e7e19b666679324a40c1b8a14c21ddde4ad5433fc2328b6933f2f1b18c0440e5. The reference execution used GNU Make 4.3 at /usr/bin/make, CPython 3.12.14, Linux 6.18.55 x86_64, glibc 2.39, and dash as the resolved recipe shell.

from future import annotations

import datetime as dt
import hashlib
import json
import os
import platform
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path

OLD = b"BEGIN\nrelease=v1\nEND\n"
GOOD = b"BEGIN\nrelease=v2\nEND\n"
PARTIAL = b"BEGIN\nrelease=v2\n"
T0 = 1_600_000_000_000_000_000
SECOND = 1_000_000_000


def build() -> int:
    mode = os.environ["BUILD_MODE"]
    target = Path("artifact.txt")
    with Path("attempts.txt").open("a", encoding="ascii") as log:
        log.write(mode + "\n")
    payload = Path("input.txt").read_bytes()
    complete = b"BEGIN\n" + payload + b"END\n"
    partial = b"BEGIN\n" + payload
    if not mode.startswith("stage"):
        previous = target.stat() if target.exists() else None
        target.write_bytes(complete if mode == "good" else partial)
        if mode == "preserve_mtime":
            assert previous is not None
            os.utime(target, ns=(previous.st_atime_ns, previous.st_mtime_ns))
        return 0 if mode in {"good", "bad_success"} else 7
    fd, name = tempfile.mkstemp(prefix=".candidate-", dir=".")
    candidate = Path(name)
    try:
        with os.fdopen(fd, "wb") as output:
            output.write(complete if mode == "stage_good" else partial)
        if mode == "stage_fail":
            return 7
        if candidate.read_bytes() != complete:
            return 65
        os.replace(candidate, target)
        return 0
    finally:
        candidate.unlink(missing_ok=True)


def snapshot(work: Path) -> dict:
    target = work / "artifact.txt"
    exists = target.exists()
    data = target.read_bytes() if exists else None
    attempts = work / "attempts.txt"
    return {
        "exists": exists,
        "bytes_hex": data.hex() if exists else None,
        "text": data.decode("ascii") if exists else None,
        "sha256": hashlib.sha256(data).hexdigest() if exists else None,
        "mtime_ns": target.stat().st_mtime_ns if exists else None,
        "input_mtime_ns": (work / "input.txt").stat().st_mtime_ns,
        "attempts": attempts.read_text().splitlines() if attempts.exists() else [],
        "candidates": sorted(p.name for p in work.glob(".candidate-*")),
    }


def main() -> None:
    if not debug:
        raise RuntimeError("HOLD: run without Python -O")
    assert os.name == "posix", "HOLD: POSIX fixture required"
    make = shutil.which("make")
    assert make, "HOLD: GNU make is required"
    make = str(Path(make).resolve())
    env = {"PATH": os.defpath, "LC_ALL": "C", "LANG": "C",
           "FIXTURE_PYTHON": sys.executable}
    version = subprocess.check_output([make, "--version"], env=env,
                                     text=True).splitlines()[0]
    assert version.startswith("GNU Make"), "HOLD: unsupported make"
    source = Path(__file__).read_bytes()
    # Independent expected bytes and attempt counts, declared before execution.
    cases = [
        ("M01", "", False, False, False,
         [("good", 0, GOOD, 1)], "ACCEPT_SYNTHETIC_TARGET"),
        ("M02", "", False, False, False,
         [("fail", 2, PARTIAL, 1), ("good", 0, PARTIAL, 1)],
         "HOLD_PARTIAL_TARGET_SKIPPED"),
        ("M03", ".DELETE_ON_ERROR:\n", False, False, False,
         [("fail", 2, None, 1), ("good", 0, GOOD, 2)],
         "REPAIR_DELETE_AND_RETRY"),
        ("M04", ".DELETE_ON_ERROR:\n", True, False, False,
         [("fail", 2, None, 1)], "HOLD_OLD_OUTPUT_LOST"),
        ("M05", ".DELETE_ON_ERROR:\n.PRECIOUS: artifact.txt\n", False, False, False,
         [("fail", 2, PARTIAL, 1), ("good", 0, PARTIAL, 1)],
         "HOLD_PRECIOUS_RETAINS_PARTIAL"),
        ("M06", ".DELETE_ON_ERROR:\n", False, True, False,
         [("fail", 0, PARTIAL, 1)], "HOLD_IGNORED_FAILURE"),
        ("M07", ".DELETE_ON_ERROR:\n", False, False, False,
         [("bad_success", 0, PARTIAL, 1)], "HOLD_ZERO_EXIT_INVALID_OUTPUT"),
        ("M08", ".DELETE_ON_ERROR:\n", True, False, False,
         [("preserve_mtime", 2, PARTIAL, 1), ("good", 0, GOOD, 2)],
         "REPAIR_MTIME_BOUNDARY_AND_RETRY"),
        ("M09", ".DELETE_ON_ERROR:\n", True, False, False,
         [("stage_fail", 2, OLD, 1), ("stage_good", 0, GOOD, 2)],
         "REPAIR_STAGED_RETRY"),
        ("M10", ".DELETE_ON_ERROR:\n", True, False, False,
         [("stage_bad", 2, OLD, 1)], "HOLD_VALIDATION_OLD_OUTPUT_PRESERVED"),
        ("M11", ".DELETE_ON_ERROR:\n", True, False, False,
         [("stage_good", 0, GOOD, 1)], "ACCEPT_STAGED_TARGET"),
        ("M12", ".DELETE_ON_ERROR:\n", True, False, True,
         [("stage_good", 2, None, 1)], "HOLD_POSTPUBLICATION_DELETE"),
    ]
    report = {
        "status": "INCOMPLETE", "started_utc": dt.datetime.now(dt.UTC).isoformat(),
        "source_sha256": hashlib.sha256(source).hexdigest(),
        "make": version, "make_path": make,
        "make_sha256": hashlib.sha256(Path(make).read_bytes()).hexdigest(),
        "python": sys.version.split()[0], "python_path": sys.executable,
        "system": platform.platform(), "recipe_shell": "/bin/sh",
        "resolved_shell": str(Path("/bin/sh").resolve()), "cases": [],
    }
    with tempfile.TemporaryDirectory(prefix="refonte-make-failure-") as temporary:
        root = Path(temporary)
        for case_id, policy, existing, ignore, postfail, steps, decision in cases:
            work = root / case_id
            work.mkdir()
            (work / "builder.py").write_bytes(source)
            (work / "input.txt").write_bytes(b"release=v2\n")
            os.utime(work / "builder.py", ns=(T0, T0))
            os.utime(work / "input.txt", ns=(T0 + 2*SECOND, T0 + 2*SECOND))
            if existing:
                (work / "artifact.txt").write_bytes(OLD)
                os.utime(work / "artifact.txt", ns=(T0 + SECOND, T0 + SECOND))
            prefix = "-" if ignore else ""
            makefile = ("SHELL := /bin/sh\n.SHELLFLAGS := -c\n" + policy
                        + "artifact.txt: input.txt builder.py\n\t" + prefix
                        + '"$$FIXTURE_PYTHON" builder.py --build\n'
                        + ("\tfalse\n" if postfail else ""))
            (work / "Makefile").write_text(makefile, encoding="ascii")
            initial = snapshot(work)
            assert initial["input_mtime_ns"] == T0 + 2*SECOND
            if existing:
                assert initial["mtime_ns"] < initial["input_mtime_ns"]
            record = {"case": case_id, "makefile": makefile, "initial": initial,
                      "steps": [], "decision": decision}
            for index, (mode, rc, wanted, attempts) in enumerate(steps):
                command = [make, "--no-print-directory", "-rR", "-j1",
                           "-f", "Makefile", "artifact.txt"]
                result = subprocess.run(command, cwd=work, env={**env, "BUILD_MODE": mode},
                                        capture_output=True, text=True, timeout=5)
                after = snapshot(work)
                assert result.returncode == rc, (case_id, result)
                assert after["bytes_hex"] == (wanted.hex() if wanted is not None else None), case_id
                assert len(after["attempts"]) == attempts, case_id
                assert after["candidates"] == [], case_id
                if wanted is not None:
                    if mode == "preserve_mtime" or wanted == OLD:
                        assert after["mtime_ns"] == initial["mtime_ns"], case_id
                    else:
                        assert after["mtime_ns"] > after["input_mtime_ns"], case_id
                record["steps"].append({"mode": mode, "command": command,
                                        "make_returncode": result.returncode,
                                        "stdout": result.stdout, "stderr": result.stderr,
                                        "after": after})
            report["cases"].append(record)
        report["status"] = "PASS_12_CASES"
    report["finished_utc"] = dt.datetime.now(dt.UTC).isoformat()
    print(json.dumps(report, indent=2))


if name == "__main__":
    if sys.argv[1:] == ["--build"]:
        raise SystemExit(build())
    main()

Run it as:

python cloud_make_failure_fixture.py > cloud_make_failure_results.json

Do not use python -O; the fixture explicitly rejects optimization because its assertions are evidence gates. Require the command exit status to be zero and the JSON to be complete with all twelve ordered cases and PASS_12_CASES. A failed assertion, timeout, missing report, truncated report, unexpected case order, or status other than PASS_12_CASES is HOLD.

Each case receives its own directory under an owned TemporaryDirectory. Python's temporary-directory documentation states that the context manager removes the temporary directory and its contents on ordinary context completion. The JSON snapshots are accumulated in memory, the owned tree is removed on normal exit, and only then is the report printed. That design deliberately means a process that stops before final reporting can lose the filesystem evidence it had collected; the fixture is a validation harness, not a durable forensic recorder.

Staged cases create a named candidate with tempfile.mkstemp(prefix=".candidate-", dir="."). The Python 3.12 tempfile documentation explains that a supplied dir controls where mkstemp() creates the file and that the caller is responsible for deleting it. The finally block performs that owned cleanup on these normal control paths; it is not a promise about uncatchable termination or every possible cleanup failure.

The preparation execution ran from 2026-10-29T11:52:01.968496+00:00 through 2026-10-29T11:52:02.757087+00:00, the controller returned 0, and the report ended at PASS_12_CASES. Treat those as supplied preparation observations. Any independent rerun should record a new source hash, executable identities, environment, and result rather than borrowing the reference identity.

Verify bytes, timestamps and invocation counts

All three measurements are necessary. Bytes distinguish absent, OLD, PARTIAL, and GOOD. Timestamps explain why ordinary make freshness logic may or may not schedule the recipe. attempts.txt proves whether the producer ran, including deliberately failing calls. A second make exit 0 is not evidence of regeneration unless the attempt count increases.

The timestamp ordering is asserted through os.utime, not estimated with sleeps. Python exposes nanosecond timestamp setting through os.utime, while the fixture reads st_mtime_ns back through stat. Hashes identify exact byte sequences, but they become meaningful only because the expected OLD, GOOD, and PARTIAL byte strings are declared independently in the source. A hash by itself does not establish correctness, provenance, or release permission. An incomplete evidence packet remains HOLD.

Compare default retention, deletion, and cleanup blind spots

M01 establishes the uncomplicated baseline. With no existing target, BUILD_MODE=good runs the producer once, make returns 0, and artifact.txt becomes GOOD. The result demonstrates that the fixed graph, working directory, helper, and byte oracle can produce the requested file under the ordinary path.

M02 is the core case of a partial target surviving a failed recipe. The first invocation uses BUILD_MODE=fail. The helper records one attempt, writes PARTIAL directly to artifact.txt, returns 7, and GNU make reports failure with process status 2. Default GNU make retention leaves PARTIAL at the target pathname. This matches the manual's warning that a failed recipe can leave a changed target whose timestamp makes it look current later.

The separate retry changes only BUILD_MODE to good. Make returns 0, but the target stays PARTIAL and the attempt list stays at one entry. That is the decisive evidence: the producer was never called a second time. The later zero means “nothing needed rebuilding under the declared freshness rules,” not “v2 was regenerated successfully.”

Nothing in M02 requires make to remember a previous successful build, and nothing depends on masking the first failure. The first invocation genuinely failed. The false green arises because the failed invocation itself left a target that later satisfied make's timestamp test.

Enable deletion and prove that the recipe reruns

M03 adds the special target declaration .DELETE_ON_ERROR:. GNU make documents that when .DELETE_ON_ERROR appears anywhere in the Makefile, make deletes the target of a rule if that target changed and the recipe exits nonzero.

The first M03 invocation again writes PARTIAL and fails. This time the post-invocation snapshot finds artifact.txt absent. On the separate retry with BUILD_MODE=good, the ordinary missing-target condition makes the rule out of date, the attempt count rises from one to two, and the new target equals GOOD. That increase is what proves regeneration. Merely seeing GOOD would not show whether it was newly produced or preserved from some earlier path.

.DELETE_ON_ERROR is a Makefile-level cleanup policy, not content-aware rollback. It reacts to a changed target plus an unignored nonzero recipe result. It does not validate application framing, understand that END\n is required, or reconstruct prior bytes. The policy gives make a better retry boundary by removing a changed failed target; it does not make each command transactional.

So M03 earns REPAIR_DELETE_AND_RETRY: repair the cleanup boundary, then perform an owned retry and verify both producer invocation and GOOD bytes. The absence created after failure is useful here because the target is wholly generated and safely reproducible. In a different project, deleting an owned target can have different consequences, so the target's ownership contract still matters.

Inspect exemptions and failures that cleanup cannot detect

M05 combines .DELETE_ON_ERROR: with .PRECIOUS: artifact.txt. The GNU manual documents .PRECIOUS primarily as preventing deletion when make is interrupted or killed and as preserving certain intermediate files. The supplied GNU Make 4.3 preparation run observed an additional interaction in this ordinary error path: PARTIAL survived the first failed invocation, and the next good-mode make returned 0 without a second producer call. That interaction between .DELETE_ON_ERROR and .PRECIOUS is an observation from GNU Make 4.3 in this fixture; it should not be silently generalized into a version-independent documentary claim.

M06 shows how ignored recipe errors affect cleanup. The generated recipe line has a leading -, so the helper can return 7 after writing PARTIAL, yet make treats the line's error as ignored and the observed make status is 0. GNU's error documentation says ignored errors are treated like success for this purpose. Because there is no unignored recipe failure, .DELETE_ON_ERROR does not rescue the content contract. PARTIAL remains and the case is HOLD.

M07 removes even the nonzero signal. The helper writes PARTIAL but deliberately returns 0 in bad_success mode. Make returns 0. .DELETE_ON_ERROR has nothing to trigger on. This is the central limit of cleanup-based policy: make can act on process status and target modification, but it does not know the file's application meaning. A content validator must exist somewhere if completeness matters.

The distinct upstream problem of preserving a producer's failure through shell wrappers is covered in preserving CI process failures. This laboratory starts after that concern: its negative controls already deliver the statuses make sees, so pipeline status propagation is intentionally outside scope.

A restored timestamp changes the deletion decision

M08 changes bytes and then restores the previous target mtime with os.utime. The helper fails, but the target remains PARTIAL rather than being deleted. That qualifies the cleanup boundary: .DELETE_ON_ERROR responds to make's changed-target detection, and GNU's interrupt-handling documentation describes deletion based on whether the target modification time changed since make first checked it.

Do not misclassify M08 as another up-to-date skip. The restored target mtime is still older than input.txt, so the separate retry sees the target as out of date, runs the producer a second time, and produces GOOD. The case shows why a target's modification time is operational evidence, but not a digest of its file content.

Preserve older output with staged publication

M04 starts with OLD already present. Because input.txt is newer, make correctly schedules the recipe. The direct-write helper opens the final pathname and writes PARTIAL, thereby destroying the previous v1 bytes before it returns 7. .DELETE_ON_ERROR then removes the changed failed target, leaving the path absent. The policy has cleaned up the partial result, but it has not restored OLD.

That is the important limitation of “delete on failure.” If the recipe overwrites a previously valid generated artifact in place, the old bytes are gone before make gets a chance to react. Deletion can turn “incomplete v2” into “no target”; it cannot infer or reconstruct “intact v1.” GNU make's manual frames deletion as ensuring a later attempt remakes the target, not as version restoration.

For some workflows, an absent generated file is the right failure state. For others, preserving valid build output is a requirement because readers may continue consuming the last accepted version while a new generation attempt fails. M04 demonstrates that direct overwrite plus cleanup cannot provide that property.

M09 and M10 move the destructive boundary. They keep OLD at the final pathname while writing a separate candidate. In M09, generation fails before publication, so OLD remains; a later staged retry validates GOOD and replaces it. In M10, generation completes only to PARTIAL, validation returns 65, and OLD again remains untouched. That preservation is about pathname publication in this controlled fixture. It does not prove that v1 satisfies the pending v2 request, and no concurrent production reader was tested.

Validate a candidate before replacing the target

The staged design publishes build artifacts through a deliberately small sequence: create a named candidate in the destination directory, write the would-be new bytes, close the file, validate its full contents, then call os.replace(candidate, target) only if validation succeeds. The Python 3.12 os.replace documentation says that an existing destination file is replaced if permitted, the operation may fail across filesystems, and a successful rename is atomic under the platform contract. On Linux, rename(2) similarly states that replacing an existing destination is atomic with respect to the destination pathname, while open file descriptors to the old file are unaffected.

M09 tests generation failure. The candidate receives PARTIAL, stage_fail returns 7 before validation or replacement, the finally block removes the named candidate, and OLD remains at artifact.txt. The separate retry calls the producer again; stage_good writes complete v2 framing, validates it against the helper's complete value, replaces the final pathname, and returns 0. The external controller then independently verifies that the final bytes equal its previously declared GOOD constant. This separation is useful: the producer's internal validator is not allowed to be the only judge of its own output.

M10 tests validation rejection. stage_bad writes PARTIAL, the explicit byte comparison fails, and the helper returns 65 before publication. Make reports failure, OLD survives, and the candidate list is empty after normal cleanup. This negative case shows why a generated file must be validated before replacement: invalid new bytes never reach the final pathname.

M11 tests the positive path. The complete candidate validates, os.replace publishes it, the helper returns 0, and the controller confirms GOOD. No incomplete candidate is promoted in M09 through M11. A retry after M09 is a fresh generation attempt; it is not a reconstruction of discarded candidate bytes.

This is also the clearest difference between cleanup and publication design. .DELETE_ON_ERROR governs what make may remove after a recipe error. Staging controls when new bytes become visible at the final pathname. The two policies can coexist, but they solve different problems. The staged approach protects OLD against pre-publication generation and validation failures. It does not protect against every failure that occurs after replacement.

Keep the candidate in the destination directory

The fixture passes dir="." to mkstemp, so the candidate is created in the same controlled working directory as artifact.txt. The Python 3.12 tempfile documentation explains both the explicit directory parameter and caller responsibility for cleanup. This supports generating a temporary file and renaming it into place from a Makefile, without adding a fallback copy path.

Keeping the candidate beside the destination also avoids the cross-filesystem case that os.replace warns can fail; Linux rename(2) reports EXDEV when old and new pathnames are on different mounted filesystems.

Bound the claim carefully. Atomic pathname replacement is not crash persistence, an fsync protocol, a transaction across multiple files, or preservation of the old target's mode, ownership, or ACLs. mkstemp creates a new file with its own metadata, so adapting this pattern to executable or policy-sensitive artifacts needs a separate metadata design. Existing open descriptors and remote-filesystem anomalies are also outside this laboratory; rename(2) explicitly notes NFS caveats.

Keep later failures out of the publication recipe

M12 is the control that prevents the staged pattern from being oversold. The helper runs stage_good: it writes complete bytes, validates them, calls os.replace, and returns 0. The generated Makefile then has a second recipe line containing false. That later command fails. Because .DELETE_ON_ERROR is active and the final target has changed, the observed post-command state is absent.

The intermediate publication is inferred from controlled program flow: the helper can return 0 in stage_good only after its validation and replacement branch completes, and make then advances to the later false line. No concurrent reader sampled artifact.txt between those events, so the fixture does not claim direct observation of a consumer reading GOOD during the window.

The engineering lesson is about recipe boundaries. Put fallible content validation before publication whenever possible. If later diagnostics, signing steps, metadata emission, or other commands can fail after publication, decide explicitly whether they belong in the same target recipe and what cleanup policy should apply. Otherwise a valid new file can be briefly published and then deleted because a subsequent recipe line fails.

This is why the staged recipe is not “transactional” merely because one rename is atomic. GNU make still evaluates the recipe as a sequence of shell invocations, and .DELETE_ON_ERROR still applies to the target when the overall recipe fails under its documented conditions. The atomic operation narrows the publication boundary; it does not turn all later work into one rollback-capable transaction.

Treat post-publication failure as a separate state

A consumer might have opened the new pathname after replacement and before the later failure, but no such reader was measured here. Automatic deletion cannot undo whatever an external consumer may already have observed or copied.

Treat “publication happened, then recipe failed” as its own recovery state. An owner must reconcile the published bytes, the final recipe status, any downstream effects, and the desired retry policy. Blindly rerunning is not sufficient evidence, and adding .PRECIOUS would merely alter retention; it would not define what downstream consumers should believe.

Reconcile the evidence and perform an owned retry

The supplied reference run on 2026-10-29 used GNU Make 4.3. “Calls” is the number of producer attempts recorded in attempts.txt, including deliberately failing calls. PASS_12_CASES means every control matched its declared expected result; for negative controls, that is a diagnostic pass, not a successful build outcome.

Case

Condition

Make status

Target after invocation(s)

Calls

Decision

M01

Direct success

0

GOOD

1

ACCEPT synthetic target

M02

Default failure, then good-mode retry

2 → 0

PARTIAL → PARTIAL

1 → 1

HOLD: producer skipped

M03

.DELETE_ON_ERROR, then retry

2 → 0

absent → GOOD

1 → 2

REPAIR, owned retry

M04

Direct overwrite of OLD fails

2

absent

1

HOLD: OLD lost

M05

Deletion plus .PRECIOUS, then retry

2 → 0

PARTIAL → PARTIAL

1 → 1

HOLD: partial retained

M06

Recipe-line error ignored

0

PARTIAL

1

HOLD: ignored failure

M07

Producer reports zero with incomplete bytes

0

PARTIAL

1

HOLD: invalid success

M08

Failure restores previous mtime, then retry

2 → 0

PARTIAL → GOOD

1 → 2

REPAIR boundary, retry

M09

Staged generation fails, then retry

2 → 0

OLD → GOOD

1 → 2

REPAIR staged retry

M10

Staged candidate fails validation

2

OLD

1

HOLD: OLD preserved

M11

Validated staged replacement

0

GOOD

1

ACCEPT synthetic target

M12

Replacement, then later recipe failure

2

absent

1

HOLD: post-publication delete

The cases divide cleanly by failure boundary. M02 demonstrates default retention and a skipped producer. M03 demonstrates actual regeneration only because deletion makes the target absent and the attempt count rises. M04 proves deletion does not restore overwritten bytes. M05 through M07 show retention or invalidity outside the successful cleanup path. M08 shows changed bytes can escape modification-time-based deletion yet still rebuild because its restored timestamp remains older than the input. M09 through M11 establish the staged pre-publication boundary. M12 proves that a later recipe failure can still remove the already replaced target.

All inspected candidate-path lists were empty after the commands returned. That supports normal-path cleanup for these cases only. No signals, power loss, parallel writers, cleanup failures, arbitrary paths, symlink containment, remote filesystems, or abrupt termination were tested.

Contain suspect outputs and perform an owned retry

When a generated target fails, the first job is not “run make again.” The first job is to identify what state the failed invocation left and whether a retry would execute anything. A practical containment procedure is:

  1. Identify the exact target pathname, its declared prerequisites, and any consumers that could promote or copy it. Record the failed command, make status, target bytes or digest, mtime, and any independent invocation witness before altering the directory.

  2. Prevent a suspect target from being promoted. PARTIAL is HOLD even if a later make command exits 0; OLD may be structurally valid but still fails the requested v2 contract.

  3. Review the cleanup and publication boundary. For wholly generated disposable targets, .DELETE_ON_ERROR may provide the desired absent-on-failure state. If retaining the previous valid version matters, prefer validated staging so failure occurs before the final pathname is replaced.

  4. Retry only when the generator's side effects are owned, repeatable, and restored to a known boundary. In this fixture that means the disposable per-case directory; in a shared project the team must define ownership separately.

  5. After retry, verify both output content and execution evidence. GOOD plus an increased producer-call count demonstrates regeneration; a zero status with unchanged calls does not.

Do not convert that into a universal make clean, recursive parent-directory deletion, or “delete everything and try again” policy. Broad cleanup can destroy unrelated evidence or artifacts and can obscure the defect that caused the false green. The repair should be as narrow as the owned target and its publication contract.

The distinction between HOLD, REPAIR, and owned retry matters operationally. HOLD stops propagation while evidence is incomplete. REPAIR changes the policy that produced the unsafe state. Owned retry is then a controlled experiment proving the repair. Acceptance comes only after the requested bytes and execution boundary agree.

Assign decisions, owners and release evidence

A build-recovery policy works better when each decision has an owner rather than being collapsed into “CI passed.” The fixture suggests a compact division:

Evidence or decision

Primary owner

Required evidence

Generator content contract

Generator owner

Declared GOOD bytes or application validator; repeatable side effects

Target cleanup/publication policy

Makefile maintainer

Rule prerequisites, .DELETE_ON_ERROR/.PRECIOUS/ignore policy, staging boundary

Retry authorization

Generator + Makefile owners

Known target state, contained consumers, proof that retry will execute

Synthetic target acceptance

Build reviewer

Expected bytes, invocation count, status, relevant timestamps

Release permission

Release authority

Independent promotion policy, provenance, approvals, broader tests

ACCEPT is appropriate only when the measured target contract and execution evidence agree: for example, M01 and M11 produce GOOD and record the expected call. REPAIR applies when the mechanism is understood but unsafe, as in default retention or a publication boundary that needs staging. HOLD applies to incomplete output, an ambiguous rerun, invalid zero-exit output, lost OLD when preservation was required, or incomplete laboratory evidence. An owned retry follows only after the condition that made retry unreliable has been corrected.

Carry forward enough evidence to prevent later reinterpretation: artifact identity or exact bytes, relevant source revision, exact invocation, make and interpreter identities when material, target/prerequisite timing evidence, producer-call evidence, case result, and the human decision. Hashes can identify artifacts and scripts, but they do not decide whether promotion is permitted.

That last separation is the subject of Refonte Learning's article on artifact provenance and release permission: verified origin and artifact identity are distinct from authorization to release. This fixture's successful content check belongs on the evidence side of that boundary, not on the approval side.

Build reviewable delivery habits through practical training

Failed-target recovery is a small example of a broader DevOps habit: define ownership, make failure states observable, keep evidence attached to the decision, and design retry boundaries before an incident forces improvisation. That fits the wider DevOps lifecycle ownership perspective, where build, integration, delivery, and monitoring are treated as connected responsibilities rather than isolated commands.

For readers who want structured practice around adjacent foundations, Refonte Learning's DevOps Engineering program lists a three-month format at 12 to 14 hours per week and includes Linux fundamentals and scripting plus CI/CD in its curriculum. The programme page also lists a capstone project; its admission section says applicants must be working toward a bachelor's or higher-level degree. These curriculum details do not establish that the programme explicitly teaches GNU make, .DELETE_ON_ERROR, or this exact Python staging fixture.

The durable engineering takeaway is narrower and more useful: after a recipe failure, inspect the bytes that survived, prove whether a later invocation ran the producer, and place cleanup and publication boundaries where the owner can reason about them. A green retry without those facts is only a process status. It is not evidence that the requested artifact was regenerated.