curl Retried Successfully. Why Does the File Contain Duplicate Rows?

A two-row CSV export finishes with curl exit status zero. The downloaded file parses, the numeric column contains reasonable values, and the automation reports success. Then a consumer notices that R1 appears twice. The intended rows are R1 and R2; the retained file contains R1, R1, and R2. Accepting that file would turn an expected total of 30 into 40.

The server dataset does not change in this laboratory, and compression plays no part. A deliberately incomplete first response contributes six bytes. A successful second response contributes twelve. Whether the first six survive depends on who controls the output destination across attempts.

This playbook follows that boundary using a synthetic, loopback-only HTTP/1.1 GET. Its reference baseline uses curl 8.10.1 and CPython 3.13.5. Those are recorded execution versions, not claims about the newest releases. There are no credentials, redirects, range requests, or state-changing application operations.

The investigation produces an attempt ledger, compares five output routes, and validates the resulting artifact against independently fixed bytes. It then adds a guarded publication design that preserves an existing accepted file when transfer or validation fails.

The operational question is whether the artifact can be handed to its consumer. A successful request retry supplies part of that evidence. Byte identity, required rows, the expected total, and the publication decision must supply the rest.

Define a complete artifact independently of curl

Establish the acceptance contract before opening a connection. The complete, headerless CSV is exactly b"R1,10\nR2,20\n": twelve bytes, two records, and a final newline. R1 and R2 must each appear once, their values must be 10 and 20, and the numeric total must be 30. Row order also belongs to this fixture's exact-byte contract.

The deliberately contaminated artifact is b"R1,10\nR1,10\nR2,20\n". It has eighteen bytes, three records, and a total of 40. All three records remain valid CSV. These numbers and identifiers are synthetic controls chosen to make the failure inspectable; they are not production measurements or metadata inferred from a download.

Calculate the trusted SHA-256 digest from the fixed source constant before making any requests. Keep that value in the run's manifest, alongside the expected length, IDs, and total. The validator later computes a digest of the candidate and compares it with this prior value. A downloaded file's own checksum is useful for identifying that file, but it cannot establish which file should have arrived.

This contract separates several questions that a green command often collapses. Did a transfer finish? Does the artifact match the intended representation? Does its record identity match the consumer's expectations? May it replace the currently published artifact? Each answer needs its own evidence, even though this small fixture allows them to be inspected together.

Keep the server's advertised response length in the attempt ledger as a separate field. It describes that response's declared body size. The independently fixed twelve-byte artifact contract describes what the consumer should receive. Recording both prevents a transfer's own declaration from becoming the only acceptance reference.

For wider context, backend and API integration foundations help place the download within its application contract. Here, keep the acceptance rule deliberately strict: no normalization, sorting, deduplication, or inferred repairs. An artifact that differs from the known twelve bytes must fail validation, including one that happens to produce a plausible total.

Start a loopback server with a deliberate partial first response

Save the following complete runner as retry_output_probe.py, and save the validator shown later as validate_artifact.py in the same directory. Prerequisites are CPython 3.13.5, a curl 8.10.1 executable, a writable working directory, and permission to bind a loopback socket. The shell examples additionally use Bash and ordinary local file utilities. No third-party Python packages or external endpoint are needed.

The server binds 127.0.0.1 on port zero, letting the operating system choose an available port. Each output case receives its own route and counter. On that route's first request, the handler advertises twelve bytes, writes the six-byte R1 prefix, flushes, and closes. On its second request, it writes all twelve advertised bytes.

Python's versioned http.server reference documents the handler, output stream, HTTP version setting, and connection-close control used here. The deliberately incorrect first response length is fault injection. This small server is a local testing instrument, not a production API server.

```python
import argparse
import json
import os
import platform
import shutil
import subprocess
import tempfile
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from validate_artifact import FULL, TRUSTED_SHA256, inspect_bytes
PREFIX = FULL[:6]
ENV = {"PATH": os.environ.get("PATH", "/usr/bin:/bin"), "LC_ALL": "C"}
OPTIONS = ["-q", "--noproxy", "*", "--http1.1", "--silent", "--show-error", "--retry", "1", "--retry-delay", "1", "--max-time", "3", "--retry-max-time", "5"]
EXPECTED = {
    "plain_retry": (18, 1, PREFIX, False),
    "redirected_stdout": (0, 2, PREFIX + FULL, False),
    "curl_output_file": (0, 2, FULL, True),
    "output_dash": (0, 2, PREFIX + FULL, False),
    "pipe": (0, 2, PREFIX + FULL, False),
}
class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"
    def log_message(self, *args):
        pass
    def do_GET(self):
        self.connection.settimeout(3)
        with self.server.lock:
            attempt = self.server.counts.get(self.path, 0) + 1
            self.server.counts[self.path] = attempt
            body = PREFIX if attempt == 1 else FULL
            advertised = len(FULL)
            # Optional extension routes: excluded from the five-case baseline.
            if self.path.startswith("/always-partial/"):
                body = PREFIX
            elif self.path.startswith("/duplicate/"):
                body = PREFIX + FULL
                advertised = len(body)
            record = {"path": self.path, "attempt": attempt, "method": "GET", "status": 200, "advertised_bytes": advertised, "body_hex": body.hex(), "flushed": False, "complete": False}
            self.server.ledger.append(record)
        try:
            self.send_response(200)
            self.send_header("Content-Length", str(advertised))
            self.send_header("Connection", "close")
            self.end_headers()
            self.wfile.write(body)
            self.wfile.flush()
            with self.server.lock:
                record["flushed"] = True
        except OSError as error:
            with self.server.lock:
                record["write_error"] = str(error)
        finally:
            self.close_connection = True
            with self.server.lock:
                record["complete"] = True
                self.server.lock.notify_all()
class Fixture:
    def __enter__(self):
        self.server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
        self.server.daemon_threads = False
        self.server.lock = threading.Condition()
        self.server.counts = {}
        self.server.ledger = []
        self.base_url = f"http://127.0.0.1:{self.server.server_port}"
        self.thread = threading.Thread(target=self.server.serve_forever)
        self.thread.start()
        return self
    def __exit__(self, *args):
        try:
            self.server.shutdown()
        finally:
            self.server.server_close()
            self.thread.join()
def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--curl", default="curl")
    parser.add_argument("--evidence", type=Path, required=True)
    parser.add_argument("--serve", action="store_true")
    args = parser.parse_args()
    curl = shutil.which(args.curl)
    if curl is None:
        raise SystemExit("RERUN: curl executable not found")
    version = subprocess.run([curl, "-q", "--version"], capture_output=True, text=True, env=ENV, check=True, timeout=5)
    curl_version = version.stdout.splitlines()[0]
    if (platform.python_implementation() != "CPython" or platform.python_version() != "3.13.5" or curl_version.split()[1] != "8.10.1"):
        raise SystemExit(f"RERUN: need CPython 3.13.5 / curl 8.10.1; found {platform.python_version()} / {curl_version}")
    args.evidence.mkdir(parents=True, exist_ok=False)
    report = {"python": platform.python_version(), "curl": version.stdout, "trusted_sha256": TRUSTED_SHA256, "cases": []}
    failed = False
    with Fixture() as fixture:
        report["base_url"] = fixture.base_url
        try:
            if args.serve:
                print(fixture.base_url, flush=True)
                try:
                    threading.Event().wait()
                except KeyboardInterrupt:
                    pass
                return 0
            with tempfile.TemporaryDirectory(prefix="refonte_retry_") as temp:
                work = Path(temp)
                try:
                    for name, (rc, attempts, expected, accepted) in EXPECTED.items():
                        output = work / f"{name}.csv"
                        cmd = [curl] + OPTIONS
                        if name != "plain_retry":
                            cmd += ["--retry-all-errors"]
                        if name == "curl_output_file":
                            cmd += ["--output", str(output)]
                        elif name == "output_dash":
                            cmd += ["--output", "-"]
                        cmd += [f"{fixture.base_url}/{name}"]
                        # Passing this open fd as stdout has the same ownership
                        # boundary as shell > file. PIPE captures a real pipe.
                        timed_out = False
                        try:
                            if name in ("curl_output_file", "pipe"):
                                process = subprocess.run(cmd, capture_output=True, env=ENV, timeout=10)
                            else:
                                with output.open("wb") as sink:
                                    process = subprocess.run(cmd, stdout=sink, stderr=subprocess.PIPE, env=ENV, timeout=10)
                        except subprocess.TimeoutExpired as error:
                            timed_out = True
                            process = subprocess.CompletedProcess(cmd, None, error.stdout or b"", error.stderr or b"")
                        if name == "pipe":
                            output.write_bytes(process.stdout)
                        data = output.read_bytes() if output.exists() else b""
                        result = inspect_bytes(data)
                        with fixture.server.lock:
                            completed = fixture.server.lock.wait_for(lambda: sum(row["path"] == f"/{name}" and row["complete"] for row in fixture.server.ledger) == attempts, timeout=3)
                            count = fixture.server.counts.get(f"/{name}", 0)
                            witness = [dict(row) for row in fixture.server.ledger if row["path"] == f"/{name}"]
                        ledger_ok = completed and [(row["attempt"], row["advertised_bytes"], row["body_hex"], row["flushed"]) for row in witness] == [(i, 12, (PREFIX if i == 1 else FULL).hex(), True) for i in range(1, attempts + 1)]
                        stderr = process.stderr.decode("utf-8", errors="replace")
                        (work / f"{name}.stderr").write_bytes(process.stderr)
                        passed = (not timed_out and process.returncode == rc and count == attempts and ledger_ok and data == expected and result["accepted"] == accepted and "curl: (18)" in stderr)
                        failed |= not passed
                        report["cases"].append({"case": name, "command": cmd, "curl_rc": process.returncode, "timed_out": timed_out, "attempts": count, "body_hex": data.hex(), "body": data.decode("ascii", errors="backslashreplace"), "stderr": stderr, "artifact": result, "evidence_file": f"files/{name}.csv", "retained_path": str((args.evidence / "files" / output.name).resolve()), "ledger_passed": ledger_ok, "test_passed": passed})
                finally:
                    shutil.copytree(work, args.evidence / "files")
        finally:
            with fixture.server.lock:
                report["requests"] = [dict(row) for row in fixture.server.ledger]
            (args.evidence / "report.json").write_text(json.dumps(report, indent=2) + "\n", encoding="utf-8")
    print(json.dumps(report, indent=2))
    return 1 if failed else 0
if __name__ == "__main__":
    raise SystemExit(main())
```

Run the five-case probe with the pinned executable selected explicitly. Use a fresh evidence directory for each run; its retained copies survive the runner's temporary-directory cleanup.

python3.13 --version
curl --version
python3.13 retry_output_probe.py \
  --curl "$(command -v curl)" \
  --evidence evidence-pinned

Make partial delivery deterministic

Nothing waits for random packet loss or a remote service to become unreliable. The failure is the server's intentional termination of a response shorter than its declared body length. The retry delay belongs to curl's scheduling; it does not create the truncation.

The request ledger records the case path, attempt ordinal, advertised length, and body supplied by the handler. Treat that as the server-side witness, then compare it with the client status and retained bytes. A server write alone does not prove which bytes reached the artifact.

A completion condition lets the runner wait for each handler's evidence update before checking the ledger. Receiving the last body byte and recording the server's final bookkeeping are separate events. This synchronization avoids a timing-dependent assertion while keeping the deliberate short response as the sole transfer fault.

The runner shuts down the serving loop, closes the server, joins its thread, and removes only its own temporary directory. Evidence remains in the explicitly requested directory. Separate extension routes support the publication tests later in this playbook. Those tests have their own execution evidence; the five-case baseline does not exercise them.

Measure the default retry control first

Start with the failure classification before changing output ownership. The control includes --retry 1, but omits --retry-all-errors. Its first response ends after six of twelve advertised bytes. In the executed curl 8.10.1 baseline, that produces one request, exit status 18, and a six-byte artifact containing only R1.

The official retry chapter describes retries for selected transient conditions and the separate opt-in for other errors. A retry count does not mean every failure is eligible. This fixture's partial-transfer error does not trigger the second request under the default retry policy.

For the shell comparisons, start the runner's server mode in one terminal. It prints the loopback URL and preserves its ledger when stopped with Ctrl-C. In a second terminal, set BASE to that exact printed URL and define the common arguments. Each command below uses a previously unused route.

Terminal 1:

python3.13 retry_output_probe.py \
  --curl "$(command -v curl)" \
  --evidence manual-server-evidence \
  --serve

Terminal 2:

read -r -p 'Paste the printed loopback URL: ' BASE
CURL_BIN=$(command -v curl)
PROBE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/curl-output-shell.XXXXXX")
printf 'Shell evidence: %s\n' "$PROBE_DIR"
CURL_ARGS=(-q --noproxy "*" --http1.1 --silent --show-error --retry 1 --retry-delay 1 --max-time 3 --retry-max-time 5)
if "$CURL_BIN" "${CURL_ARGS[@]}" "$BASE/shell-plain" \
    >"$PROBE_DIR/plain.csv" 2>"$PROBE_DIR/plain.stderr"; then
    curl_rc=0
else
    curl_rc=$?
fi
printf '%s\n' "$curl_rc" >"$PROBE_DIR/plain.rc"
printf 'curl_rc=%s\n' "$curl_rc"
wc -c <"$PROBE_DIR/plain.csv"
cat "$PROBE_DIR/plain.stderr"

Expected control evidence is separate from the command:

curl_rc=18
server_requests=1
retained_bytes=6
retained_ids=R1

Capture stderr even though the failure is intentional. The official exit-code reference identifies code 18 as a partial-file transfer. That status explains why this attempt failed; it does not establish that the retained prefix is a usable export. R2 is absent, the total is only 10, and the trusted digest cannot match.

Keep -q as curl's first argument to prevent a personal configuration file from silently changing the invocation. --noproxy "*" keeps this local exercise away from configured proxies, and --http1.1 fixes the protocol choice. --silent --show-error suppresses the progress meter while retaining error diagnostics. The bounded retry and timeout values remain identical across the comparisons.

The control is useful because it establishes the partial artifact before retries concatenate anything. It also prevents the mistaken explanation that the server spontaneously supplied three rows. For the separate curl POST redirect validation problem, request routing is the subject. Here every request remains a GET to its assigned loopback route, and the next experiment changes only error eligibility.

Both fixture responses begin with HTTP status 200. The first still produces curl error 18 because its declared body is incomplete. Therefore an HTTP status field, a curl process result, and the artifact validation result belong in different parts of the evidence record. None should be renamed simply “success” without specifying which boundary it describes.

Reproduce duplicate rows with redirected stdout

Add --retry-all-errors and send stdout to a fresh file using shell redirection. The destination on disk is a regular file, but the shell opens it and passes the resulting descriptor to curl as stdout. The pathname is absent from curl's output configuration.

if "$CURL_BIN" "${CURL_ARGS[@]}" --retry-all-errors \
    "$BASE/shell-redirected" >"$PROBE_DIR/redirected.csv" \
    2>"$PROBE_DIR/redirected.stderr"; then
    curl_rc=0
else
    curl_rc=$?
fi
printf '%s\n' "$curl_rc" >"$PROBE_DIR/redirected.rc"
printf 'curl_rc=%s\n' "$curl_rc"
wc -c <"$PROBE_DIR/redirected.csv"
cat "$PROBE_DIR/redirected.stderr"
cat "$PROBE_DIR/redirected.csv"

The first attempt writes R1,10 followed by a newline and then fails. Curl performs the allowed retry. The second attempt writes the complete two-row body to the same stdout stream. In the executed reference case, the final curl status is zero, there are two server requests, and the file contains eighteen bytes:

R1,10
R1,10
R2,20

The retry duplicated the output even though the shell used > rather than an append operator. The shell opened the file once for the whole curl process. It does not reopen or truncate that destination every time curl internally starts another HTTP attempt. Both bodies therefore reach the same continuing output stream.

The descriptor may refer to an operating-system file that supports seeking. The claim is narrower than saying such a descriptor is inherently unseekable: curl does not reset this stdout destination during the tested retry workflow. Output ownership, rather than the filename's appearance or its location on disk, determines the observed behavior.

The curl man page's retry warning explicitly distinguishes a curl-managed output file from redirected stdout and pipes. It warns that failed-attempt data already written through redirection is not reset and can produce duplicates. The rolling manual describes the boundary; the pinned fixture measures its consequences for these exact bytes.

Keep the first attempt visible in the evidence

The partial-transfer diagnostic remains in stderr even after the successful retry. Preserve the original line from this pinned case:

curl: (18) end of response with 6 bytes missing

This is the runtime error diagnostic. The longer warning about redirected output belongs to curl's documentation; it should not be invented as an additional line in this command's log. With the selected silent settings, do not require an informational retry message to prove that another request occurred. The server ledger supplies that count.

Keep the final process status, command arguments, both request entries, complete stderr, byte count, decoded IDs, and digest together. A summary that retains only the last successful request discards the evidence explaining the duplicate. A summary that labels all stderr as a final command failure loses the opposite distinction: curl did eventually return zero.

The file must still be rejected. Its contents violate the fixed artifact contract in several independent ways: length, row multiplicity, total, and digest. No inference about network reliability is necessary. The eighteen retained bytes can be reconciled directly as the six-byte failed prefix followed by the twelve-byte successful response. Changing the status label cannot change that arithmetic.

Give curl ownership of a regular output file

Repeat the same first-partial, second-complete schedule with a new route and --output stage.csv. Supply a fresh pathname as curl's own option, keep stdout separate from the CSV, and use neither append nor resume behavior.

if "$CURL_BIN" "${CURL_ARGS[@]}" --retry-all-errors \
    --output "$PROBE_DIR/stage.csv" "$BASE/shell-named" \
    2>"$PROBE_DIR/named.stderr"; then
    curl_rc=0
else
    curl_rc=$?
fi
printf '%s\n' "$curl_rc" >"$PROBE_DIR/named.rc"
printf 'curl_rc=%s\n' "$curl_rc"
wc -c <"$PROBE_DIR/stage.csv"
cat "$PROBE_DIR/stage.csv"

The reference run makes two requests and finishes with status zero, just as the contaminated stdout case does. This time the resulting regular file contains twelve bytes: one R1 record followed by one R2 record. Its total is 30 and its digest matches the value derived before any request.

The important change is who manages the output across attempts. With this pathname route, curl can reset the failed attempt's written portion before receiving the next body. The first six bytes are therefore removed from the candidate's retained representation. The successful retry supplies the complete replacement body instead of following the failed prefix.

That behavior is consistent with the project's version-tagged retry documentation. Reading the tagged documentation also prevents a rolling manual from quietly expanding the syntax assumed by a curl 8.10.1 example. The repair uses flags available in that version, including the existing --output option.

Keep the conclusion bounded to the tested destination: a fresh regular file opened by curl, one process, and the specified GET schedule. A pipe, a special file, a shared pathname with another writer, or a different transfer mode would need its own evidence. The presence of an output-related flag alone is not a sufficient ownership test.

Named output also does not turn the live publication path into a suitable staging area. Failed attempts can exist in that file while the process is running, and an unsuccessful final transfer can leave an unacceptable candidate. Use a private staging path that consumers do not read. Transfer completion and the content validator must both authorize the later handoff.

For this case the ownership change repairs the observed duplication mechanism, and the independent validator supplies artifact acceptance. Retain both facts. Reporting only that --output worked would hide the contract against which the resulting twelve bytes were judged.

Test output destinations that only look like the repair

Two similar-looking commands retain the original problem. The first spells the destination as --output - and redirects stdout to disk. The second sends stdout through a pipe to a sink. Both routes produced two requests, final curl status zero, and eighteen retained bytes in the executed baseline.

if "$CURL_BIN" "${CURL_ARGS[@]}" --retry-all-errors --output - \
    "$BASE/shell-dash" >"$PROBE_DIR/dash.csv" \
    2>"$PROBE_DIR/dash.stderr"; then
    curl_rc=0
else
    curl_rc=$?
fi
printf '%s\n' "$curl_rc" >"$PROBE_DIR/dash.rc"
printf 'dash curl_rc=%s\n' "$curl_rc"
wc -c <"$PROBE_DIR/dash.csv"
if (
    set -o pipefail
    "$CURL_BIN" "${CURL_ARGS[@]}" --retry-all-errors "$BASE/shell-pipe" \
        2>"$PROBE_DIR/pipe.stderr" | cat >"$PROBE_DIR/pipe.csv"
); then
    pipeline_rc=0
else
    pipeline_rc=$?
fi
printf '%s\n' "$pipeline_rc" >"$PROBE_DIR/pipe.rc"
printf 'pipeline_rc=%s\n' "$pipeline_rc"
wc -c <"$PROBE_DIR/pipe.csv"

A hyphen selects stdout. It does not give curl a filename that it can reset between attempts. The shell still controls the file behind the redirected descriptor, so the route has the same attempt-to-stream relationship as the earlier > example. Naming an option --output does not change what its argument means.

The pipe case makes the boundary more visible. Bytes leave curl and enter another process's input stream. The sink writes what arrives: first the partial R1 row, then the complete R1/R2 body. By the time curl decides to retry, a downstream program may already have stored or interpreted the prefix. This fixture's sink simply retains it, which makes the evidence easy to inspect.

The Python runner uses subprocess.PIPE to exercise the same stream boundary without depending on a particular shell sink. It captures stdout as bytes and subsequently saves those bytes to the case file. That later write is owned by Python. It cannot retroactively turn the earlier pipe into a file that curl controlled during retry.

Do not confuse that with the runner's redirected-stdout cases. There, Python opens a regular file before starting curl and supplies its descriptor as stdout, matching the relevant ownership of shell redirection. In the named-output case, Python instead passes the pathname in curl's argument list. The comparison changes the ownership at the point where rollback matters.

Why pipefail does not clean the bytes

Failure propagation remains useful when a pipeline component fails. The related article on preserving failure status through tee addresses that separate responsibility. Here, however, the final curl status really is zero, and a sink that successfully writes eighteen bytes can also return zero.

Consequently, enabling pipefail does not necessarily produce a failing pipeline in this experiment. It cannot retract the failed prefix or tell the sink which rows belong to which attempt. An artifact validator can reject the resulting stream contents after capture, but that is an additional decision based on content.

Keep the consumer behind that decision. If a downstream process acts on each row while curl is still retrying, a later rejection of the completed file arrives after those rows have already been exposed. For this playbook, the sink stores evidence only. Publication remains a distinct step after the complete staged artifact passes its contract.

Compare attempts, outcomes, and file contents together

The five rows below are the executed curl 8.10.1 / CPython 3.13.5 reference baseline. Every case begins with its own partial first response. The four opt-in cases receive a successful second response; the default retry control stops after the first.

All routes require R1 and R2 exactly once. The table's ID column records what was actually retained against that requirement. The default control behaved as configured. Its RERUN outcome requires a justified recovery decision and a still-valid source contract; it is not an instruction to enable all-error retries everywhere.

Read across each row before comparing rows. Two requests establish that the retry happened. Status zero establishes the final process outcome. Twelve or eighteen bytes identify different artifacts. The IDs and total explain the consumer-visible consequence, while the trusted digest anchors the exact representation. None of these columns substitutes for the others.

The five-case Python probe printed here was executed with locally built curl 8.10.1 and CPython 3.13.5. Its top-level assertions confirmed every table row. All nine request-ledger entries matched the planned bodies and completed flushes. The standalone validator also accepted the correct file and rejected both damaged forms. The separate publication tests described below ran on the same pinned runtimes and passed their three controlled cases.

The general warning about output reset comes from curl's documentation. The exact request counts, bodies, totals, and statuses come from this controlled schedule. Neither form of evidence establishes behavior for every curl release or every download mode.

Readers investigating why gzip changes download length comparisons need a different representation analysis. Here the bytes are uncompressed throughout. The eighteen-byte result is completely explained by retaining data from two attempts, so a length mismatch should lead directly to attempt reconciliation and artifact rejection.

Validate the staged artifact before promotion

Save this complete validator as validate_artifact.py. It checks exact bytes and the prior digest, requires both IDs exactly once, counts rows, and verifies the numeric total. It returns zero for accepted fixture content and 65 for an unreadable or rejected artifact. Transfer success remains a separate prerequisite checked by the caller.

import csv
import hashlib
import io
import json
import sys
from collections import Counter
from pathlib import Path
FULL = b"R1,10\nR2,20\n"
# The oracle exists before any request or downloaded file.
TRUSTED_SHA256 = hashlib.sha256(FULL).hexdigest()
def inspect_bytes(data):
    errors, rows, total = [], [], None
    digest = hashlib.sha256(data).hexdigest()
    try:
        rows = list(csv.reader(io.StringIO(data.decode("ascii"), newline=""), strict=True))
        if any(len(row) != 2 for row in rows):
            raise ValueError("each row must contain two fields")
        total = sum(int(row[1]) for row in rows)
    except (UnicodeError, csv.Error, ValueError) as error:
        errors.append(f"CSV rejected: {error}")
    ids = [row[0] for row in rows if row]
    if len(data) != len(FULL):
        errors.append("expected 12 bytes")
    if len(rows) != 2 or Counter(ids) != Counter({"R1": 1, "R2": 1}):
        errors.append("expected exactly one R1 row and one R2 row")
    if total != 30:
        errors.append("expected numeric total 30")
    if data != FULL or digest != TRUSTED_SHA256:
        errors.append("bytes or trusted digest mismatch")
    return {"accepted": not errors, "bytes": len(data), "ids": ids, "total": total, "sha256": digest, "errors": errors}
def main():
    if len(sys.argv) != 2:
        print("usage: python3.13 validate_artifact.py FILE", file=sys.stderr)
        return 64
    try:
        result = inspect_bytes(Path(sys.argv[1]).read_bytes())
    except OSError as error:
        print(f"artifact read failed: {error}", file=sys.stderr)
        return 65
    print(json.dumps(result, indent=2))
    return 0 if result["accepted"] else 65
if __name__ == "__main__":
    sys.exit(main())

The CSV parser supplies structure, not permission to publish. The contaminated R1/R1/R2 body parses without difficulty. Its rejection comes from the contract: three rows instead of two, repeated R1, eighteen bytes instead of twelve, an incorrect total, and a mismatched digest. The function reports these discrepancies without deleting or rewriting any record.

Use the preserved named-output artifact as the positive control and a preserved stdout artifact as the negative control. The executed validator checks returned zero for the twelve-byte body and 65 for both the six-byte prefix and eighteen-byte duplicate body. Treat an unexpected acceptance of either negative control as a failed test of the validator itself.

Acceptance requires both a successful current transfer and correct current content. A valid file left by an older invocation cannot excuse today's failed download. Fresh staging paths and run-specific evidence bind the candidate to the operation being reviewed, so a passing digest cannot accidentally authorize a stale file.

python3.13 validate_artifact.py evidence-pinned/files/curl_output_file.csv || exit 1
if python3.13 validate_artifact.py evidence-pinned/files/redirected_stdout.csv; then
    printf 'FAIL: duplicate artifact was admitted\n' >&2
    exit 1
else
    validator_rc=$?
    test "$validator_rc" -eq 65 || exit 1
fi

Checks that overlap are still useful when they explain different failure consequences. The digest establishes byte identity with the oracle. The IDs and total show why the candidate is wrong for the consumer. If R1 and R2 exchanged their numeric values, a count of two rows and a total of 30 could still pass; the exact-byte comparison would reject that different representation.

Use an oracle that did not come from the suspect download

For this fixture, the trusted source is the literal FULL defined before the server starts. Its independently calculated SHA-256 is:

2a9339afd32bc40e1c61537f69c3b93c9500398898349be2fbec0952036f641e

In an operational workflow, agree on an independently trusted manifest or application contract tied to a stable export identity. It must identify what the consumer expects, including any representation details relevant to byte-level checks. Do not replace that reference with a digest calculated only from the suspect download, or with a second unverified download of unknown content.

A matching digest here means equality to the known fixture bytes under the chosen check. It does not establish a broader authenticity claim about an unrelated service. Likewise, a mismatch is a reason to reject and investigate, not permission to adjust the reference until the artifact passes. Keep the original expected contract and the measured candidate digest together so another engineer can repeat the comparison.

Publish the accepted file without exposing failed attempts

The next file, publish_fixture.py, downloads into a fresh private staging directory beneath the destination's existing parent, captures evidence, checks curl's status, validates the candidate, and only then attempts replacement. Three executed controls verified accepted publication, rejection after final transfer failure, and rejection of complete but duplicated content. Both rejection controls preserved the previous destination's exact bytes.

```python
import json
import os
import platform
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from urllib.parse import urlsplit
from retry_output_probe import ENV, OPTIONS
from validate_artifact import inspect_bytes
def main():
    if len(sys.argv) != 5:
        print("usage: publish_fixture.py CURL URL DESTINATION EVIDENCE", file=sys.stderr)
        return 64
    executable, url, target, evidence_name = sys.argv[1:]
    curl = shutil.which(executable)
    parsed = urlsplit(url)
    if (curl is None or parsed.scheme != "http" or parsed.hostname != "127.0.0.1" or parsed.username is not None or parsed.password is not None):
        print("RERUN: require curl and a credential-free loopback HTTP URL", file=sys.stderr)
        return 23
    version = subprocess.run([curl, "-q", "--version"], capture_output=True, text=True, env=ENV, check=True, timeout=5)
    if (platform.python_implementation() != "CPython" or platform.python_version() != "3.13.5" or version.stdout.split()[1] != "8.10.1"):
        print("RERUN: require CPython 3.13.5 and curl 8.10.1", file=sys.stderr)
        return 23
    destination = Path(target).absolute()
    evidence = Path(evidence_name).absolute()
    evidence.mkdir(parents=True, exist_ok=False)
    report = {"python": platform.python_version(), "curl": version.stdout, "destination": str(destination), "published": False}
    # Parent must already exist; the caller owns it and supplies one writer.
    with tempfile.TemporaryDirectory(prefix=".download-", dir=destination.parent) as temp:
        stage = Path(temp) / "stage.csv"
        cmd = [curl] + OPTIONS + ["--retry-all-errors", "--output", str(stage), url]
        report["command"] = cmd
        try:
            process = subprocess.run(cmd, capture_output=True, env=ENV, timeout=10)
            report["curl_rc"] = process.returncode
            stderr = process.stderr
        except subprocess.TimeoutExpired as error:
            report.update(curl_rc=None, timed_out=True)
            stderr = error.stderr or b""
        data = stage.read_bytes() if stage.exists() else b""
        (evidence / "candidate.csv").write_bytes(data)
        (evidence / "curl.stderr").write_bytes(stderr)
        report["artifact"] = inspect_bytes(data)
        if report["curl_rc"] != 0:
            status = 20
        elif not report["artifact"]["accepted"]:
            status = 21
        else:
            status = 0
        report["gate_rc"] = status
        # Save transfer and content evidence before the namespace changes.
        (evidence / "report.json").write_text(json.dumps(report, indent=2) + "\n")
        if status:
            return status
        try:
            os.replace(stage, destination)
        except OSError as error:
            report.update(gate_rc=22, promotion_error=str(error))
        else:
            report["published"] = True
        try:
            (evidence / "report.json").write_text(json.dumps(report, indent=2) + "\n")
        except OSError:
            print("HOLD: reconcile destination and unfinished publication evidence", file=sys.stderr)
            return 24
        return report["gate_rc"]
if __name__ == "__main__":
    raise SystemExit(main())
```

Supply a fixture URL, an owned destination, and a new evidence directory. The destination's parent must already exist. The wrapper enforces the same pinned versions and restricts requests to loopback HTTP. Its fixed digest makes this a fixture wrapper; adapting it to another export requires that export's independently established contract.

mkdir -p "$PROBE_DIR/publication"
if python3.13 publish_fixture.py "$CURL_BIN" "$BASE/publish-shell" \
    "$PROBE_DIR/publication/current.csv" "$PROBE_DIR/publish-evidence"; then
    printf 'Accepted fixture publication completed\n'
else
    gate_rc=$?
    printf 'Publication gate returned %s; inspect its evidence\n' "$gate_rc" >&2
    exit "$gate_rc"
fi

Placing the staging directory under the destination's parent keeps the replacement on the same filesystem. Python's os.replace documentation describes replacement and the atomic rename property when the operation succeeds, while noting that crossing filesystems can fail. For this design, atomicity concerns the pathname handoff. It does not supply disk durability, crash recovery, or coordination among competing writers.

The preservation rule is specific: transfer failure or content rejection does not reach replacement, so the existing last-good destination remains in place. The two rejection controls verified that rule for their modeled failures. Failed candidates and stderr are copied into the evidence directory before staging cleanup. Consumers should read the published destination only, with one designated writer controlling this handoff.

A separate local failure can occur after replacement, while finalizing its evidence record. The wrapper reports that uncertainty as HOLD rather than claiming the old file survived. Inspect the actual destination and retained evidence before another operation. The filesystem replacement and the JSON audit update are not one transaction.

On normal exit, the temporary staging directory removes only its own remaining files. The evidence directory remains available. This preserves the distinction between cleaning working space and destroying the material needed to explain a rejected candidate or an uncertain publication state.

Bound retries without hiding the actual outcome

The small timing values make the local experiment manageable. --retry 1 permits one retry after the initial attempt, subject to eligibility and timing limits. Thus the successful retry cases make two requests, while the default partial-transfer control makes only one. Increasing the count would not repair output ownership; it could give more failed attempts an opportunity to contribute bytes.

The version-tagged retry-time documentation explains why the five-second retry budget is not a universal process deadline. A transfer started before the retry limit can continue beyond it. --max-time 3 limits an individual attempt, and --retry-delay 1 supplies the pause before another eligible attempt. The runner separately applies a subprocess timeout to keep the laboratory bounded.

Use the integer values shown. The rolling manual documents newer fractional retry-delay and retry-budget syntax that the pinned fixture does not need. The execution claim remains curl 8.10.1 with CPython 3.13.5. Reading current documentation does not silently upgrade those runtimes.

--retry-all-errors is an explicit fault-recovery choice for this controlled, idempotent GET. It is needed to make the deliberately induced partial-transfer error retryable in the comparison. It should not become an unconditional production default. Decide eligibility, attempt limits, output ownership, and artifact validation together for the actual download contract.

Separate a failed transfer from a failed content check

The wrapper gives its gate a separate status while retaining the underlying curl status in the report:

A timeout record uses an absent curl result plus a timeout flag instead of inventing a native curl exit code. A content rejection after curl zero remains a failing wrapper result. Automation must propagate that gate result to its caller; printing a reassuring completion message must not override it. The publication controls exercised statuses 0, 20, and 21. They did not inject timeouts, replacement errors, or failure of the final evidence write. Unexpected local setup or I/O exceptions can also terminate Python with a nonzero result outside these explicit wrapper statuses.

Reconcile a suspect artifact without silently rewriting it

Start reconciliation by freezing the evidence, not by editing the CSV. Keep the original file, its measured digest, the full invocation, runtime versions, stderr, the request ledger, and the expected contract. Assign a quarantine location that consumers cannot mistake for the published artifact. The suspect file must remain byte-for-byte available for analysis.

For the eighteen-byte fixture, the ledger provides a direct explanation: attempt one supplied the six-byte R1 prefix, and attempt two supplied the complete twelve-byte body. The retained file contains both contributions in order. That evidence supports repairing the output route and rejecting the candidate. It does not support publishing a manually shortened version of that candidate.

Outside this controlled schedule, a repeated ID alone does not establish retry concatenation as the cause. Keep that explanation provisional until the attempt history and retained byte sequence support it. The source could have supplied a duplicate itself; the independent contract would still reject it, but the responsible repair would differ.

Deleting one R1 row might appear harmless in this tiny example because the correct bytes are already known. It would nevertheless test a different repair: data rewriting after contamination. The required repair is to prevent failed-attempt bytes from surviving in the new staged artifact. Keep the original candidate untouched and demonstrate that mechanism with a fresh run.

Give the rerun a new evidence directory as well as a new candidate pathname. Overwriting the earlier report would erase the comparison that shows why the repair was needed. Link the two runs through their common expected contract, while retaining their distinct invocation, route, process result, and measured artifact identity.

Use a simple operational sequence:

1. Record the current publication identity before investigating the candidate.

2. Preserve the candidate and all available attempt evidence under one run identity.

3. Compare its bytes, IDs, total, and digest with the previously trusted contract.

4. Change the download route to a private, curl-managed regular staging file.

5. Rerun against the same established source identity and capture a new ledger.

6. Authorize publication only from the new candidate's complete acceptance evidence.

Within the laboratory, a new route resets the attempt schedule while the FULL constant remains unchanged. Reusing a route that already received its second request would skip the intended partial-first response on a later command. That could produce an apparently clean rerun without exercising the repair. Preserve route identity as part of the command evidence.

In a real integration, a new URL request does not by itself establish an unchanged source representation. If the export identity has changed, obtain the corresponding trusted manifest and treat it as a new candidate. Comparing a new export blindly with the old digest can explain a rejection incorrectly; replacing the old digest without establishing the new identity can hide a defect.

Stop the manual loopback server with Ctrl-C so its final ledger is written and its socket closes. Retain the shell case directory printed during setup until its evidence has been reviewed or copied to the intended handoff location. Any later cleanup should target that exact fixture-owned directory, never a broad filename pattern that might include production artifacts.

Apply a download acceptance matrix

Assign the outcome to the artifact and its evidence package. The command's status remains one input. In particular, a clean final HTTP attempt does not authorize a contaminated file, and an expected negative control does not become an accepted artifact merely because the test harness succeeded.

RERUN is useful when the missing evidence can be re-established. It must not disguise the absence of any trustworthy source contract. If nobody can establish which representation is intended, another successful download cannot resolve the acceptance question; HOLD remains appropriate.

Test the wrapper as well as curl

Save the following publication test as test_publish_fixture.py. The positive case checks publication of the correct bytes. The all-attempts-fail case always advertises twelve bytes and sends six, so the wrapper must retain the previous destination after curl's final failure. A separate complete-but-duplicate response advertises and sends eighteen bytes; curl can succeed while the wrapper must reject its content.

The previous-publication marker in this test is synthetic and deliberately differs from the new candidate. It represents content already accepted under a prior contract. Comparing its exact bytes after each rejection makes accidental replacement visible; checking only that the destination still exists would miss that failure entirely.

# Publication extension: run explicitly; it is not part of the five-case probe.
import json
import subprocess
import sys
import tempfile
from pathlib import Path
from retry_output_probe import Fixture, PREFIX
from validate_artifact import FULL
def require(condition, message):
    if not condition:
        raise AssertionError(message)
if len(sys.argv) != 3:
    raise SystemExit("usage: test_publish_fixture.py CURL NEW_EVIDENCE_DIRECTORY")
curl = sys.argv[1]
audit = Path(sys.argv[2]).absolute()
audit.mkdir(parents=True, exist_ok=False)
wrapper = Path(__file__).with_name("publish_fixture.py")
last_good = b"previously-accepted-fixture\n"
cases = [
    ("accepted", "/publish/good", 0, 0, 2, FULL, FULL),
    ("all_fail", "/always-partial/fail", 20, 18, 2, last_good, PREFIX),
    ("bad_content", "/duplicate/reject", 21, 0, 1, last_good, PREFIX + FULL),
]
with tempfile.TemporaryDirectory(prefix="publish_test_") as temp:
    root = Path(temp)
    with Fixture() as fixture:
        try:
            for name, route, gate_rc, curl_rc, attempts, expected, candidate in cases:
                target = root / f"{name}.csv"
                target.write_bytes(last_good)
                evidence = audit / f"{name}-evidence"
                process = subprocess.run([sys.executable, str(wrapper), curl, fixture.base_url + route, str(target), str(evidence)], capture_output=True, timeout=15)
                (audit / f"{name}.stdout").write_bytes(process.stdout)
                (audit / f"{name}.stderr").write_bytes(process.stderr)
                (audit / f"{name}.published.csv").write_bytes(target.read_bytes())
                require(process.returncode == gate_rc, f"{name}: wrong wrapper status")
                require(target.read_bytes() == expected, f"{name}: wrong published bytes")
                record = json.loads((evidence / "report.json").read_text())
                require((evidence / "candidate.csv").read_bytes() == candidate, f"{name}: wrong candidate evidence")
                require(record["curl_rc"] == curl_rc, f"{name}: lost curl status")
                require(record["published"] == (gate_rc == 0), f"{name}: wrong handoff record")
                with fixture.server.lock:
                    count = fixture.server.counts.get(route, 0)
                require(count == attempts, f"{name}: wrong request count")
        finally:
            with fixture.server.lock:
                completed = fixture.server.lock.wait_for(lambda: all(row["complete"] for row in fixture.server.ledger), timeout=3)
                ledger = [dict(row) for row in fixture.server.ledger]
            (audit / "requests.json").write_text(json.dumps(ledger, indent=2) + "\n")
            require(completed, "incomplete server ledger")
print("PASS: accepted bytes published; both failure controls preserved last-good")

python3.13 test_publish_fixture.py \
  "$(command -v curl)" publication-test-evidence

The top-level test must fail if duplicates are admitted, a partial candidate replaces the destination, or either modeled rejection changes last-good content. It must also fail if the wrapper loses curl's underlying status or records the wrong number of attempts. The executed test process returned zero because these assertions passed. The accepted case made two requests, returned curl status 0 and wrapper status 0, and published twelve bytes. The all-fail case made two requests, returned curl 18 and wrapper 20, and retained a rejected six-byte candidate. The complete duplicate case made one request, returned curl 0 and wrapper 21, and retained eighteen rejected bytes. Its curl stderr was empty. Both failures left the last-good destination unchanged.

Each case uses a separate output pathname and evidence directory. Candidate bytes, wrapper output, and the request ledger survive outside temporary working space. The fixture closes its server and cleans its temporary destinations on exit. These controls concern the specified transfer and content failures; they do not claim to test disk faults, power loss, hostile local writers, or every possible publication failure.

Assign ownership from HTTP attempt to consumer handoff

The handoff becomes reviewable when each owner supplies evidence for a defined boundary. The API owner defines the expected representation and a stable export identity. That owner explains what constitutes the complete dataset and how a consumer obtains the matching trusted contract. The download owner should not have to infer these facts from whichever bytes arrived.

The download owner controls the curl executable, argument list, retry policy, output route, and staging location. Their evidence connects individual HTTP attempts to the retained candidate. They must know whether a wrapper changed stdout routing, whether a file was opened by curl, and where failed candidates are retained for investigation.

The consumer owner defines the meaningful completeness checks: required IDs, multiplicity, values, and other application-specific conditions. For this fixture, those checks are intentionally small and exact. In a real contract, the same owner must decide what an allowed representation change means before a validator begins accepting it.

The release owner controls the publication decision. That owner verifies transfer success and artifact acceptance together, checks that the candidate belongs to the intended source identity, and owns the destination replacement procedure. They also define what happens when the artifact is rejected or the final publication state is uncertain.

Keep a compact handoff record containing the source identity, trusted manifest identity, executable versions, exact command arguments, request ledger, candidate path and digest, validation findings, and final outcome. Record the last-good publication identity separately from the new candidate. A consumer should not need to reconstruct whether a path holds evidence, staging content, or an accepted export.

Revalidate after changes to output routing, wrapper code, curl version, or the data contract. For example, replacing a named output option with a logging pipeline can restore the failure while preserving a familiar filename. Keeping the same command text while changing the executable version also changes the evidence required for acceptance.

Work on broader API security and scalability practices also benefits from clear ownership. This handoff's specific obligation is to prove which complete artifact the consumer received. A working one-liner without an established manifest and retained validation evidence is insufficient to make that obligation reviewable.

Build stronger API integration and testing habits

Request retry success and artifact completeness answer different questions. This laboratory makes that distinction observable with a failed six-byte prefix, a successful twelve-byte response, and output destinations that retain different final bytes. The useful habit is to carry the evidence across every boundary until the consumer receives an accepted artifact.

Build that habit into ordinary integration work. Define the expected representation before downloading, make failed-attempt behavior reproducible, preserve the command and request ledger, validate staged content, and authorize publication explicitly. When a result fails, choose REPAIR, RERUN, or HOLD from the evidence instead of manufacturing a passing file. Reserve ACCEPT for a candidate that meets the complete contract.

For readers developing these skills, Refonte Learning's APIs Developer Fundamentals page lists API documentation and testing, error handling, and logging within a three-month program at 10–12 hours per week. Those published subjects provide relevant context for disciplined integration work; they do not verify instruction in this exact curl command or laboratory.

Inspect the program's testing and error-handling coverage if it fits your learning goals. For the download workflow itself, retain the practical standard demonstrated here: retry the permitted request, validate the resulting artifact independently, and hand over only the content that passed the agreed acceptance gate.