A COPY FROM can finish successfully after discarding rows. That is not a contradiction: PostgreSQL 17 introduced ON_ERROR ignore specifically so text or CSV input rows containing data-type conversion failures can be skipped while the command continues. The same documentation says successful COPY returns a COPY count command tag for rows copied, and emits a NOTICE with the number of ignored rows when at least one row was discarded. A green command status therefore answers “did this permitted best-effort load finish?” It does not by itself answer “did the database receive every row the release contract requires?”
The operational question is narrower: given COPY ... ON_ERROR ignore, can you account for every admitted input identity, explain every rejected row, prove that staging and the final table contain the authorized population, and decide whether the result is safe to release? The answer requires more than a copied-row count. It needs an independently authored case manifest, source hashes, command and warning capture, tuple-level reconciliation, controlled positive and negative cases, and an explicit owner for every discrepancy.
The database fixture below was NOT EXECUTED in the research environment. A preflight run on November 7, 2026 found pg_config reporting PostgreSQL 17.11 (Debian 17.11-0+deb13u1), while both psql and the postgres server executable were unavailable. That is not sufficient evidence of a runnable PostgreSQL 17.11 server, so every database outcome below is labeled EXPECTED / NOT EXECUTED. The supplied fixture was authored and hashed, with a recorded SHA-256 of 13352166f9764fba65dddd78c4afbc9b4ea841adcfa613f868cd9a535d640046.
Define the Operational Contract and Release Gate
The release contract must be written before looking at COPY output. For this fixture, one authorized import contains five identities, keyed by request_key, with an expected id and qty attached to each key. Success means the final table contains exactly those tuples, no more and no fewer. A skipped row is therefore an explained ingestion event, not automatically an acceptable business outcome.
This distinction follows directly from PostgreSQL's contract. ON_ERROR ignore handles errors “converting a column's input value into its data type”; the affected input row is discarded and processing continues. The option is limited to COPY FROM in text or CSV formats. PostgreSQL also documents that COPY FROM invokes destination triggers and check constraints, which matters because relational validation is a different boundary from input conversion.
The gate therefore evaluates five separate artifacts:
Evidence | Release question | Pass condition |
Source hash | Did we test the exact file under review? | Hash retained with case |
COPY status and notices | How many rows were loaded or skipped? | Status and warnings preserved verbatim |
Stage reconciliation | Did every expected tuple reach typed staging? | No missing or extra tuples |
Final-table reconciliation | Did promotion preserve the same authorized population? | No missing, extra, duplicated, or changed identities |
Transaction receipt | Did the accepted final state commit as one owned unit? | Explicit commit followed by independent reread |
A COPY 4 plus one skip notice against a five-row contract is not ACCEPT simply because 4 + 1 = 5. The arithmetic explains physical processing; it does not establish that discarding the fifth identity was authorized. A valid gate must name the missing identity and its recovery owner.
Refonte Learning's PostgreSQL ID Watermarks Can Skip Committed Rows examines a separate operational problem. That page tests a concurrency and commit-order failure in which a numeric watermark can advance past a later-committing lower ID. Its oracle reconciles a reader's closed population. The present problem is different: one COPY invocation can intentionally discard conversion-invalid input while returning success.
Use these dispositions consistently:
Decision | Meaning here |
ACCEPT | Exact expected population reconciles in final state and required evidence is complete |
REPAIR | Cause and ownership are known; input or workflow must be corrected before acceptance |
RERUN | Corrected or incomplete evidence requires the controlled case to execute again |
HOLD | Ownership, expected population, runtime evidence, or failure semantics remain unresolved |
Because the fixture was not run against a PostgreSQL server during this research, the overall laboratory decision is HOLD: NOT EXECUTED. The playbook provides expected case decisions, not invented observed ones.
Capture the Baseline and Construct a Disposable Controlled Fixture
The exact server matters. PostgreSQL 17 is the first major release whose release notes list COPY ... ON_ERROR ignore, LOG_VERBOSITY, and pg_stat_progress_copy.tuples_skipped. PostgreSQL 18's current documentation additionally includes REJECT_LIMIT and a silent LOG_VERBOSITY level, so those later options must not be projected backward onto a PostgreSQL 17 run.
The supplied fixture deliberately version-locks itself to PostgreSQL 17.x. It records SHOW server_version, server_version_num, lc_messages, client encoding, DateStyle, and the psql client version before creating measured data. If server_version_num does not begin with 17, the run stops with HOLD rather than silently applying a PostgreSQL 17 expectation to another branch.
That version lock is stronger than recording pg_config. pg_config describes installed development/build information; it does not prove that a matching server is running, that a client exists, or that the test connection reaches that server. The failed local preflight in this research environment is exactly why the article does not convert a library/build artifact into a fake execution record.
The fixture uses a dedicated schema called copy_reconcile_lab and refuses to reuse it. That makes the ownership boundary measurable: if the schema already exists, the run stops rather than dropping an unknown object. The database itself must also be disposable or isolated. No production endpoint, shared staging database, background writer, trigger not included in the fixture, replication pipeline, or external side effect belongs inside the acceptance claim.
The expected population is authored separately from the malformed CSV. In particular, the skip case says that req-c is expected with quantity 30 even though the candidate source contains not_an_int. That is deliberate. An oracle generated from the file being tested would merely restate the defect.
For related preparation, see Refonte Learning's guide, PostgreSQL IF NOT EXISTS: Validate the Actual Schema.
Executable fixture
Save the following as copy_reconcile_lab.sh, make it executable, set DATABASE_URL to a disposable PostgreSQL 17 database, and run it from a directory where the test account may create the lab schema. The recorded source hash is:
13352166f9764fba65dddd78c4afbc9b4ea841adcfa613f868cd9a535d640046#!/usr/bin/env bash
set -euo pipefail
: "${DATABASE_URL:?Set DATABASE_URL to a disposable PostgreSQL 17 database}"
LAB_SCHEMA="copy_reconcile_lab"
ARTIFACT_DIR="${ARTIFACT_DIR:-./copy_reconcile_artifacts}"
mkdir -p "$ARTIFACT_DIR"
cat > "$ARTIFACT_DIR/case_clean.csv" <<'CSV'
id,request_key,qty
101,req-a,10
102,req-b,20
103,req-c,30
104,req-d,40
105,req-e,50
CSV
cat > "$ARTIFACT_DIR/case_skip.csv" <<'CSV'
id,request_key,qty
101,req-a,10
102,req-b,20
103,req-c,not_an_int
104,req-d,40
105,req-e,50
CSV
cat > "$ARTIFACT_DIR/case_repaired.csv" <<'CSV'
id,request_key,qty
101,req-a,10
102,req-b,20
103,req-c,30
104,req-d,40
105,req-e,50
CSV
cat > "$ARTIFACT_DIR/case_duplicate.csv" <<'CSV'
id,request_key,qty
201,dup-a,10
202,dup-b,20
203,dup-b,30
204,dup-d,40
205,dup-e,50
CSV
cat > "$ARTIFACT_DIR/case_extra_field.csv" <<'CSV'
id,request_key,qty
301,extra-a,10
302,extra-b,20,unexpected
303,extra-c,30
CSV
cat > "$ARTIFACT_DIR/expected_manifest.csv" <<'CSV'
case_name,id,request_key,qty
clean,101,req-a,10
clean,102,req-b,20
clean,103,req-c,30
clean,104,req-d,40
clean,105,req-e,50
skip,101,req-a,10
skip,102,req-b,20
skip,103,req-c,30
skip,104,req-d,40
skip,105,req-e,50
repaired,101,req-a,10
repaired,102,req-b,20
repaired,103,req-c,30
repaired,104,req-d,40
repaired,105,req-e,50
CSV
sha256sum "$0" "$ARTIFACT_DIR"/*.csv |
tee "$ARTIFACT_DIR/sha256.txt"
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 -Atqc \
"SHOW server_version;
SHOW server_version_num;
SHOW lc_messages;
SHOW client_encoding;
SHOW DateStyle;" |
tee "$ARTIFACT_DIR/server_settings.txt"
psql --version | tee "$ARTIFACT_DIR/client_version.txt"
server_num="$(
psql "$DATABASE_URL" -X -Atqc 'SHOW server_version_num'
)"
if [[ ! "$server_num" =~ ^17[0-9]{4}$ ]]; then
echo "HOLD: fixture is version-locked to PostgreSQL 17.x; \
observed server_version_num=$server_num" >&2
exit 20
fi
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 <<SQL
DO \$\$
BEGIN
IF EXISTS (
SELECT 1
FROM pg_namespace
WHERE nspname = '$LAB_SCHEMA'
) THEN
RAISE EXCEPTION 'schema $LAB_SCHEMA already exists';
END IF;
END\$\$;
CREATE SCHEMA $LAB_SCHEMA;
CREATE TABLE $LAB_SCHEMA.stage (
id bigint,
request_key text,
qty integer
);
CREATE TABLE $LAB_SCHEMA.final (
id bigint PRIMARY KEY,
request_key text NOT NULL UNIQUE,
qty integer NOT NULL CHECK (qty > 0)
);
CREATE TABLE $LAB_SCHEMA.expected (
case_name text NOT NULL,
id bigint NOT NULL,
request_key text NOT NULL,
qty integer NOT NULL,
PRIMARY KEY (case_name, request_key)
);
\copy $LAB_SCHEMA.expected
FROM '$ARTIFACT_DIR/expected_manifest.csv'
WITH (FORMAT csv, HEADER true)
SQL
run_copy_case() {
local case_name="$1"
local file="$2"
local log="$ARTIFACT_DIR/${case_name}.copy.log"
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 2>&1 <<SQL |
tee "$log"
TRUNCATE $LAB_SCHEMA.stage;
\copy $LAB_SCHEMA.stage (id, request_key, qty)
FROM '$file'
WITH (
FORMAT csv,
HEADER true,
ON_ERROR ignore,
LOG_VERBOSITY verbose
)
SELECT 'stage_count', count(*)
FROM $LAB_SCHEMA.stage;
TABLE $LAB_SCHEMA.stage
ORDER BY id, request_key;
SQL
}
reconcile_stage() {
local case_name="$1"
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 -P pager=off <<SQL
WITH e AS (
SELECT id,request_key,qty
FROM $LAB_SCHEMA.expected
WHERE case_name='$case_name'
),
s AS (
SELECT id,request_key,qty
FROM $LAB_SCHEMA.stage
),
missing AS (
SELECT 'missing' AS issue, e.* FROM e
EXCEPT ALL
SELECT 'missing', s.* FROM s
),
extra AS (
SELECT 'extra' AS issue, s.* FROM s
EXCEPT ALL
SELECT 'extra', e.* FROM e
)
SELECT FROM missing
UNION ALL
SELECT FROM extra
ORDER BY issue,id,request_key;
SQL
}
# C0 positive control.
run_copy_case clean "$ARTIFACT_DIR/case_clean.csv"
reconcile_stage clean |
tee "$ARTIFACT_DIR/clean.reconcile.log"
# C1 conversion-error case.
run_copy_case skip "$ARTIFACT_DIR/case_skip.csv"
reconcile_stage skip |
tee "$ARTIFACT_DIR/skip.reconcile.log"
# C2 equal-count / wrong-identity negative control.
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 <<SQL |
tee "$ARTIFACT_DIR/negative_control.log"
BEGIN;
DELETE FROM $LAB_SCHEMA.stage
WHERE request_key='req-d';
INSERT INTO $LAB_SCHEMA.stage(id,request_key,qty)
VALUES (999,'req-x',40);
SELECT count(*) AS equal_count_after_identity_corruption
FROM $LAB_SCHEMA.stage;
WITH e AS (
SELECT id,request_key,qty
FROM $LAB_SCHEMA.expected
WHERE case_name='skip'
),
s AS (
SELECT id,request_key,qty
FROM $LAB_SCHEMA.stage
)
SELECT 'missing' AS issue, e.* FROM e
EXCEPT ALL
SELECT 'missing', s.* FROM s
UNION ALL
SELECT 'extra' AS issue, s.* FROM s
EXCEPT ALL
SELECT 'extra', e.* FROM e
ORDER BY issue,id,request_key;
ROLLBACK;
SQL
# C3 corrected source and final promotion.
run_copy_case repaired \
"$ARTIFACT_DIR/case_repaired.csv"
reconcile_stage repaired |
tee "$ARTIFACT_DIR/repaired.stage.reconcile.log"
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 <<SQL |
tee "$ARTIFACT_DIR/repaired.final.log"
BEGIN;
TRUNCATE $LAB_SCHEMA.final;
INSERT INTO $LAB_SCHEMA.final(id,request_key,qty)
SELECT id,request_key,qty
FROM $LAB_SCHEMA.stage
ORDER BY id;
WITH e AS (
SELECT id,request_key,qty
FROM $LAB_SCHEMA.expected
WHERE case_name='repaired'
),
f AS (
SELECT id,request_key,qty
FROM $LAB_SCHEMA.final
),
diff AS (
(
SELECT 'missing' AS issue,e.* FROM e
EXCEPT ALL
SELECT 'missing',f.* FROM f
)
UNION ALL
(
SELECT 'extra' AS issue,f.* FROM f
EXCEPT ALL
SELECT 'extra',e.* FROM e
)
)
SELECT
FROM diff
ORDER BY issue,id,request_key;
COMMIT;
SELECT 'final_count',count()
FROM $LAB_SCHEMA.final;
TABLE $LAB_SCHEMA.final
ORDER BY id;
SQL
# C4 relational-constraint boundary.
run_copy_case duplicate \
"$ARTIFACT_DIR/case_duplicate.csv"
set +e
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 \
>"$ARTIFACT_DIR/duplicate.final.log" 2>&1 <<SQL
BEGIN;
TRUNCATE $LAB_SCHEMA.final;
INSERT INTO $LAB_SCHEMA.final(id,request_key,qty)
SELECT id,request_key,qty
FROM $LAB_SCHEMA.stage
ORDER BY id;
COMMIT;
SQL
duplicate_rc=$?
set -e
echo "duplicate_final_rc=$duplicate_rc" |
tee -a "$ARTIFACT_DIR/duplicate.final.log"
if [[ "$duplicate_rc" -eq 0 ]]; then
echo "HOLD: duplicate control unexpectedly succeeded" >&2
exit 30
fi
# C5 structural COPY boundary.
set +e
psql "$DATABASE_URL" -X -v ON_ERROR_STOP=1 \
>"$ARTIFACT_DIR/extra_field.copy.log" 2>&1 <<SQL
TRUNCATE $LAB_SCHEMA.stage;
\copy $LAB_SCHEMA.stage (id, request_key, qty)
FROM '$ARTIFACT_DIR/case_extra_field.csv'
WITH (
FORMAT csv,
HEADER true,
ON_ERROR ignore,
LOG_VERBOSITY verbose
)
SQL
extra_rc=$?
set -e
echo "extra_field_rc=$extra_rc" |
tee -a "$ARTIFACT_DIR/extra_field.copy.log"
if [[ "$extra_rc" -eq 0 ]]; then
echo "HOLD: extra-field control unexpectedly succeeded" >&2
exit 31
fi
echo \
"NOT CLEANED: retain $LAB_SCHEMA and $ARTIFACT_DIR for review"The script intentionally retains both database state and files. Deleting evidence automatically at successful exit would make later reconciliation harder and could erase precisely the material required to explain a discrepancy. Cleanup is therefore an explicit owner action after acceptance.
Define an Independent Oracle and Controlled Cases
The manifest is an oracle only if its expected rows are approved independently of the candidate load result. Copying SELECT * FROM stage into the expected table after the run would make the database its own specification. Parsing the malformed CSV to generate expected values would have the same circular defect.
The expected table therefore contains authored tuples for clean, skip, and repaired. The skip case expects five rows even though its physical input can produce only four typed rows under ON_ERROR ignore. That disagreement is the point of the test: the loader's permitted behavior and the release contract do not have to be the same.
EXCEPT ALL is used instead of a count-only comparison so multiplicity and full tuple identity remain visible. A stage count of five could still conceal one missing row and one unauthorized replacement. The negative control proves that boundary by deleting req-d, inserting req-x, keeping the same row count, and then requiring the oracle to report both the missing and extra tuple.
That design principle is deliberately similar to, but technically separate from, Refonte's SQLite REPLACE vs UPSERT: Preserve Related Rows article. That page demonstrates that equal counts cannot prove preservation of SQLite parent/child identities after REPLACE; this fixture applies the same evidence discipline to a different mechanism, engine, and release question. It does not reproduce the SQLite test or decision.
The six cases are independently declared before execution:
Case | Input condition | Expected database behavior | Oracle expectation | Expected decision |
C0 | Five valid rows | COPY succeeds with five staged rows | No diff | ACCEPT |
C1 | One invalid integer | Four rows staged; bad row skipped | req-c missing | REPAIR |
C2 | Artificial equal-count wrong identity | Count preserved | one missing + one extra | HOLD negative control |
C3 | Corrected five-row source | Stage and final exact | No diff | ACCEPT |
C4 | Duplicate business key reaches unconstrained stage | Stage succeeds; final constraint rejects promotion | No final acceptance | REPAIR |
C5 | Extra CSV field | COPY fails despite ON_ERROR ignore | Stage must not be treated as accepted | RERUN after source repair |
All six entries are EXPECTED / NOT EXECUTED in this report.
The C0 positive control matters because a reconciliation query that reports discrepancies for the valid source is itself suspect. You should not infer that a red test proves bad input until the same collector has demonstrated a clean case. Conversely, C2 is the negative control: if the collector declares equality after an expected identity is replaced with req-x, the oracle is too weak for release acceptance.
The comparison grain is the complete (id, request_key, qty) tuple. request_key is the business identity; id and qty remain protected attributes. An implementation that compares only request_key would miss wrong IDs or changed quantities. An implementation that compares only row count would miss C2 entirely.
Reproduce the Misleading Success and Its Failure Boundaries
C1 is the central PostgreSQL COPY ON_ERROR ignore case. Its third data row supplies not_an_int for an integer column. PostgreSQL 17 documents that an input-conversion error can be handled by discarding the row and continuing; with verbose logging it emits per-row information identifying the input line and failed column, followed by a summary notice for skipped rows.
The PostgreSQL source tree's regression test exercises the same feature with malformed integer and array values. The test uses ON_ERROR ignore, LOG_VERBOSITY verbose, expects individual “skipping row due to data type incompatibility” notices, and then an aggregate skipped-row notice. PostgreSQL's source code increments the internal error count and reports skipped tuples to COPY progress reporting.
For this fixture, the expected, not observed C1 evidence is therefore:
EXPECTED / NOT EXECUTED
source data rows: 5
COPY command status: success
COPY count: 4
skipped conversion rows: 1
stage identities: req-a, req-b, req-d, req-e
oracle result: missing (103, req-c, 30)
decision: REPAIRThe important result is the oracle line, not the green command status. The stage contains exactly the four rows PostgreSQL was allowed to materialize. It still fails the independently authored five-identity contract.
Do not repair that condition by inventing a row from the notice text. The notice identifies a conversion failure; it is not a trustworthy substitute for the authorized payload. Recovery requires an owner to supply or approve the corrected req-c value. After correction, C3 reruns the complete candidate source rather than inserting one ad hoc row into an otherwise accepted stage. That produces a fresh, single-source evidence trail.
C4 demonstrates another boundary. The stage deliberately has PostgreSQL types but no business constraints. Five conversion-valid rows can therefore enter staging even when two use dup-b. Promotion into final should fail because request_key is declared UNIQUE NOT NULL. PostgreSQL's constraint documentation states that a unique constraint rejects duplicate non-null values, while the transaction documentation establishes that the failing transaction can be rolled back as an all-or-nothing unit.
That case prevents an important overgeneralization: ON_ERROR ignore is not “ignore any bad row anywhere in the write path.” Its documented scope is conversion of fields into their target data types. PostgreSQL separately says COPY FROM invokes check constraints and triggers. A duplicate key, business check, trigger exception, malformed row shape, or later final-table constraint therefore requires evidence appropriate to that failure, not a fabricated “rejected-row” count.
C5 provides a structural negative case. PostgreSQL's own regression tests mark missing-data and extra-data COPY rows as failures even when ON_ERROR ignore is specified; the expected regression output shows extra data after last expected column as an error rather than a soft row skip.
So the expected C5 result is a nonzero client exit status, not COPY 2 plus one rejection. If your executed PostgreSQL 17 build behaves differently, do not rewrite the manifest to match it. Record the exact version, raw output, source hash, and contradiction, then HOLD until the version-specific behavior is explained.
PostgreSQL also exposes pg_stat_progress_copy.tuples_skipped while a COPY is running. The PostgreSQL 17 progress-reporting documentation defines that field as the number of tuples skipped because they contain malformed data and says it advances when ON_ERROR is not stop. That view is useful operational telemetry, but because it describes a currently running COPY, it is not a durable post-run audit ledger by itself.
Capture Complete Per-Case Evidence and Reconcile Identities
A useful evidence ledger is case-oriented rather than run-oriented. Do not store “the import passed” as one Boolean. Record enough evidence to reproduce why a case was classified.
Use this ledger shape:
Field | Example purpose |
case_id | Stable test identity such as C1 |
status | EXPECTED, OBSERVED, or CONTRADICTED |
fixture_sha256 | Exact script under review |
source_sha256 | Exact candidate CSV |
manifest_sha256 | Exact independent oracle |
server_version | Full runtime server string |
server_version_num | Machine-checkable branch/patch identity |
client_version | psql identity |
lc_messages | Explains warning text/localization assumptions |
copy_exit_code | Client-level success/failure |
copy_command_tag | Rows reported copied |
notice_text | Raw rejection evidence |
stage_count | Diagnostic count only |
missing_tuples | Oracle discrepancy |
extra_tuples | Oracle discrepancy |
final_diff | Final-table discrepancy |
transaction_outcome | COMMIT, ROLLBACK, or aborted |
owner | Source, DBA, migration, or application owner |
decision | ACCEPT / REPAIR / RERUN / HOLD |
The fixture's startup sha256sum writes one receipt containing the script and every generated source file. Preserve that file before modifying anything. A repaired CSV is a new artifact and must receive a new hash. It must not inherit the rejected file's evidence merely because its filename is similar.
The stage reconciliation is deliberately tuple-based:
WITH e AS (
SELECT id, request_key, qty
FROM copy_reconcile_lab.expected
WHERE case_name = 'skip'
),
s AS (
SELECT id, request_key, qty
FROM copy_reconcile_lab.stage
)
SELECT 'missing', e.* FROM e
EXCEPT ALL
SELECT 'missing', s.* FROM s
UNION ALL
SELECT 'extra', s.* FROM s
EXCEPT ALL
SELECT 'extra', e.* FROM e;A clean result is an empty result set. That is stronger than COUNT(*) = 5, but it is still bounded: it proves equality only across the columns included in the oracle, for the retained population at that point in the transaction history. It says nothing about omitted metadata, downstream replicas, backups, or later writes.
The same reconciliation must run against final. Staging equality is necessary but not sufficient because the promotion step introduces another failure boundary. PostgreSQL INSERT is a new statement governed by target constraints, and the final table can reject a population that staging admitted.
C3 therefore does not stop after successful COPY. It begins an explicit transaction, clears the disposable final table, inserts the reconciled staged rows, computes a final EXCEPT ALL difference, and commits only the intended unit.
After commit, the fixture prints the final table. A stronger commissioning variant should open a separate read-only connection after the writer commits and rerun the same final reconciliation there. That extra reader would separate “the writer saw its transaction” from “a new transaction sees the committed state.” PostgreSQL's transaction tutorial documents that uncommitted changes are not visible to other sessions and that committed changes become visible as a unit.
This evidence discipline overlaps conceptually with Refonte's sqlite3 executemany: Recover the Right Batch IDs, but not in subject matter. That article asks whether Python sqlite3 can provide a reliable per-request generated-ID mapping after executemany(). It uses keyed reconciliation because a scalar lastrowid cannot describe a batch. The present playbook asks whether PostgreSQL's explicitly skipped COPY rows can be reconciled against an authorized population before promotion.
Retain failure artifacts before rerunning. If C1 is repaired, the C1 log, source hash, notice, stage diff, and decision remain immutable evidence. The repaired run becomes C3 or a new revision such as C3-r2. Reusing the original case identifier and overwriting its output destroys the audit trail that explains why repair was required.
Separate Documented Behavior From Observation and Repair Safely
A release record should visibly distinguish four epistemic classes.
Documented contract is what an applicable PostgreSQL primary source promises. Examples include the conversion-error scope of ON_ERROR ignore, the COPY command tag, the skip notice, destination constraint execution, transactional rollback, and the meaning of pg_stat_progress_copy.tuples_skipped.
Test hypothesis is what the fixture predicts from those contracts. “C1 should stage four rows and report one skipped conversion row” is a hypothesis until it is run on the named version. “C5 should fail on an extra field” is also a hypothesis, albeit one strongly supported by PostgreSQL's regression suite.
Observation is raw evidence from an executed fixture: exit status, warning text, command tag, selected tuples, version output, source hash, and transaction receipt. There are no PostgreSQL database observations in this report because the fixture could not be executed here.
Inference is the engineering conclusion drawn from documented behavior plus observations. For example, if an executed C1 produced four rows and the oracle reported only req-c missing, the inference would be that the best-effort load behaved as configured but failed the five-row release contract. The database did not “lose” an unexplained row; it deliberately discarded one conversion-invalid input. Acceptance still fails because the contract did not authorize omission.
Repair must preserve these distinctions. Fixing not_an_int to 30 is a source repair only if the responsible data owner confirms that 30 is the authorized value. A DBA cannot derive that business value from PostgreSQL's skip notice. Likewise, rewriting the manifest from five expected tuples to four after seeing the outcome would be a contract change, not a technical repair.
Once the source is corrected, rerun from an empty stage. Do not insert the missing tuple manually unless the release contract explicitly defines a separately auditable compensating procedure. A fresh full load avoids a hybrid state in which four tuples came from one source hash and the fifth came from an operator command outside the original ingestion path.
Rollback safety also requires a clear ownership boundary. PostgreSQL transactions group several statements into an all-or-nothing unit; a ROLLBACK cancels the transaction's changes.
That is why the final promotion is wrapped in BEGIN/COMMIT. If the duplicate-key C4 insertion fails, the transaction is not an invitation to retry a random subset. Record the failing source and constraint evidence, end the failed unit appropriately, repair the duplicate mapping with its owner, clear the disposable stage, and rerun.
This recovery boundary is fundamentally different from Refonte's MySQL DDL Implicit Commits: What ROLLBACK Cannot Undo. That article examines MySQL DDL implicitly ending a transaction so that a later rollback cannot undo earlier work. It explicitly labels its own lab unexecuted. PostgreSQL's transaction behavior in this playbook is not a portability claim about MySQL, and the PostgreSQL fixture should not import that article's DDL test.
PostgreSQL 18 introduces an additional control, REJECT_LIMIT, which can fail a COPY after more than the configured number of conversion errors. That option is valuable when a best-effort policy still has an explicit maximum tolerance, but it is intentionally excluded from this version-locked PostgreSQL 17 fixture. The PostgreSQL 18 documentation defines REJECT_LIMIT; PostgreSQL 17's COPY syntax does not list it.
Do not backport that policy concept by parsing skipped notices and pretending it is server-enforced REJECT_LIMIT. On PostgreSQL 17, an application may choose to reject a completed load after reviewing its evidence, but that is an application release gate, not the same database feature.
Apply the Acceptance Matrix, Preserve Ownership, and Build Related Skills
The final decision should be mechanical enough that two reviewers looking at the same ledger reach the same disposition.
Condition | Decision | Owner action |
Exact stage and final equality; complete runtime and artifact evidence; committed final state | ACCEPT | Publish run receipt |
Known malformed value; expected identity absent; authorized correction available | REPAIR | Correct source, hash it, rerun complete case |
Corrected source exists but no execution receipt yet | RERUN | Execute from clean stage and reconcile again |
Missing manifest, unknown source provenance, missing runtime version, unexplained contradiction, or ambiguous ownership | HOLD | Resolve evidence/ownership before writes |
Count matches but identities differ | HOLD | Reject count-only acceptance |
Stage is exact but final constraint fails | REPAIR | Resolve relational conflict; do not weaken oracle silently |
COPY structurally fails on malformed CSV shape | RERUN after repair | Correct source format and rerun from clean state |
Applied to the unexecuted fixture, the expected case ledger is:
Case | Expected evidence | Expected release result | Execution status |
C0 | COPY 5, no diff | ACCEPT | NOT EXECUTED |
C1 | COPY 4, one conversion skip, req-c missing | REPAIR | NOT EXECUTED |
C2 | Same row count but req-d missing and req-x extra | HOLD negative control | NOT EXECUTED |
C3 | COPY 5, empty stage diff, empty final diff, commit | ACCEPT | NOT EXECUTED |
C4 | Stage load succeeds; final duplicate constraint rejects promotion | REPAIR | NOT EXECUTED |
C5 | COPY exits nonzero for extra field | RERUN after source repair | NOT EXECUTED |
The laboratory release decision is therefore HOLD: DATABASE FIXTURE NOT EXECUTED. That is not a failure of the protocol. It is the correct classification when executable PostgreSQL evidence is absent.
Rerun the fixture unchanged on the target PostgreSQL 17 patch, retain sha256.txt, server_settings.txt, client_version.txt, every *.copy.log, every reconciliation log, final transaction output, and the final-table reread. Compare the executed script hash with the recorded source hash, 13352166f9764fba65dddd78c4afbc9b4ea841adcfa613f868cd9a535d640046. A changed hash is not automatically wrong, but it is a different experiment and requires a new execution record.
If an observed result contradicts an expected result, record the contradiction instead of editing it away. Capture the exact server patch and locale first; then re-check the applicable PostgreSQL version documentation and source regression behavior. Current PostgreSQL documentation on November 7, 2026 is the PostgreSQL 18 branch, while this fixture targets PostgreSQL 17, so the versioned documentation is the controlling reference for execution-specific claims.
For engineers who want broader DBA practice around the same operational disciplines, Refonte Learning's Database Administrator Essentials page states a three-month period, an expected dedication of 12–14 hours per week, basic programming knowledge recommended, and competencies including database design, SQL query optimization, backup and recovery, performance tuning, security, migration, disaster recovery, RBAC, and monitoring. The page also states that successful completion can result in a Training Certificate and Certificate of Internship.
For this PostgreSQL release gate, the standard remains narrower than any course description: identify the exact source, execute against the exact runtime, account for every skipped row, reconcile every expected identity through staging and final state, preserve the evidence, and issue ACCEPT only when the database state satisfies the independently written contract.a
