Why a 20% Group Split Can Hold Out 80% of Your Rows

Five groups supply twenty synthetic rows. Groups A, B, C, and D each own one row; group E owns sixteen. Ask GroupShuffleSplit for a 20% test size and it must select one of the five groups. If that group is E, the split holds out one fifth of the groups but sixteen twentieths, or 80%, of the rows.

Those headline numbers are the bounds of one deliberately constructed fixture. They are not an incidence rate, a probability, a prediction about production data, or evidence that group splitting usually produces an 80% row holdout. The counterexample exists to expose a unit error: GroupShuffleSplit.test_size describes unique groups, not source rows.

The operational question is therefore not “Did .2 run?” It is: Which groups and row identities entered each partition, and do those facts satisfy the approved dataset contract? The reproduction below fixes twenty row IDs, two group-size patterns, one split per case, and literal known-answer manifests. It separately evaluates group count, group isolation, source-row completeness, and an exact-four-row diagnostic.

That separation matters. A split can be correct under the documented scikit-learn API yet make a report saying “20% of rows were held out” false. Conversely, a row-based splitter can select exactly four rows while dividing one entity between train and test. The review decision must preserve the intended sampling unit rather than changing it merely to make a percentage cell green.

1. Define what the holdout percentage must measure

Before choosing a splitter, write the split contract in nouns and counts. The minimum record names the eligible source-row IDs, the group identity attached to every row, the reason groups must remain intact, the requested number or fraction of test groups, any separate requirement on test rows, and the person authorized to resolve a conflict. This is the practical layer beneath efforts to define what an evaluation split must represent.

An “exact 20% row requirement” means exactly four of this fixture’s twenty rows. A descriptive statement is different: it reports whatever row fraction the approved group split actually produced. A range such as 15%–25% is different again; it is a predeclared acceptance band whose owner and rationale must exist before candidate partitions are inspected. The laboratory uses exact-four-rows as a diagnostic because it makes the unit mismatch obvious. It does not establish that every project needs four rows, exactly 20%, or any particular range.

The intended new-entity claim also belongs in the contract. If a customer, machine, patient, device, or source is the independent held-out unit, all rows with that group identity must remain together. “Twenty rows were partitioned” does not prove that condition. Neither does a model score.

Write requirements and observations in different fields. required_test_groups=1 is a policy input; observed_test_groups=["E"] is evidence. required_test_rows=4 is another policy input; observed_test_rows=16 is evidence. A reviewer can then see whether the implementation failed, the report was mislabeled, or the declared requirements are mutually incompatible. Without that separation, teams often turn an observation into a requirement after the result appears.

Give the group and row denominators separate names

Use explicit fractions:

• test_groups / source_groups

• test_rows / source_rows

Record the numerators beside the percentages. In this fixture, 1 / 5 = 20% groups may coexist with 1 / 20 = 5% rows or 16 / 20 = 80% rows.

Keep at least four acceptance fields: requested_group_count_satisfied, overlap_groups, source partition completeness, and the row-size decision. A single split_ok Boolean would hide whether a rejection came from crossing an entity, losing a row, selecting the wrong number of groups, or missing a separately approved row quota.

2. Read the size parameter in the correct API

Parameter names do not define units by themselves. The four official pages cited in this article were checked on October 12, 2026, and displayed scikit-learn 1.8.0. That date identifies the research record; the behavior is established API semantics, not a newly introduced 2026 feature. In the versioned scikit-learn 1.8 GroupShuffleSplit documentation, test_size and train_size explicitly refer to groups rather than samples. A floating-point test_size is the proportion of groups assigned to test, rounded up; an integer is an absolute number of test groups. With train_size=None, the train side becomes the complement of the test size. An integer random_state makes repeated calls reproducible under that recorded environment.

That contract differs from scikit-learn 1.8 ShuffleSplit, whose size arguments refer to samples, and from the familiar scikit-learn 1.8 train_test_split, which wraps a sample-oriented shuffle split and applies the resulting indices to its inputs. Neither interface provides the group-isolation contract tested here merely because it also accepts an argument named test_size.

This distinction should be explicit when teams move from ordinary training, validation and test roles to grouped observations. A familiar .2 spelling is not portable evidence of a common unit; the class receiving it determines whether it counts rows or groups. The full fixture passes the groups vector, fixes n_splits=1, and leaves train_size=None. Under those bounded choices, checking that train and test are disjoint and together contain every source position is a valid complementary-partition invariant. Do not generalize that invariant to configurations where train and test sizes are independently constrained and some observations can intentionally remain unused.

Explain fractional rounding and integer group counts

Five unique groups make the arithmetic visible. ceil(.2 × 5) = 1, so .2 selects one group. ceil(.21 × 5) = 2, so .21 selects two groups, a realized group fraction of 2 / 5 = 40%. Integer test_size=1 also means one group.

These are fixture calculations, not population statistics. Whole groups are indivisible, so the realized group fraction can differ from the requested fraction even before row sizes enter the discussion. Row mass then depends on which complete groups were selected.

3. Establish the source manifest before splitting

The unequal fixture has twenty unique row identities, r00 through r19, and this literal group inventory:

The equal comparison keeps the same row identities but assigns four consecutive rows to each of A, B, C, D, and E. Retaining the same IDs prevents the equal-size control from quietly becoming a different source population. The feature matrix is only np.zeros((20, 1)); it carries no predictive meaning. There is no target vector, estimator, fit, score, class distribution, or preprocessing step. The experiment is about allocation units and identity evidence.

The commissioning record supplied for this article reports Python 3.12.14, NumPy 2.3.5, scikit-learn 1.8.0, and Linux-6.18.44-x86_64-with-glibc2.39. Its normal run began at 2026-10-12T06:58:21.614841+00:00, ended at 2026-10-12T06:58:21.634729+00:00, exited 0, and recorded PASS_10_CASES. The record also reports source SHA-256 bf56da369ed3f40dc201734dc6c20c9347adda2f202054ce42c72b7408e7c2f3 for the artifact used in that run. That digest is part of the supplied record; it is not a byte-for-byte verification of code copied from this Word article. The timestamps identify evidence, not performance measurements. No package installation or model training was reported for the commissioning run.

A new run must record its own interpreter, NumPy and scikit-learn versions, platform, start and finish times, output path, source bytes, and hash. A modified file cannot inherit the commissioning hash or its result.

Use an independent known-answer manifest

Expected test IDs are authored as literals for C1–C6. They are not generated by calling the splitter and then accepting whatever it returned. The audit also proves that train and test positions are individually unique, mutually disjoint, and exactly cover range(20).

That independence is essential. len(train) + len(test) == 20 would still pass if one row were duplicated and another omitted. Likewise, a correct total test count would not identify which entity was crossed. Every accepted complementary split must account for every declared source identity exactly once and preserve the literal row-to-group mapping.

4. Reproduce the complete ten-case experiment

Save the following supplied listing as group_shuffle_fixture.py in a writable local directory. Record the digest of the exact file you execute. Copying from Word, changing line endings, or making any edit changes the source bytes, so the commissioning digest should not be assigned to a re-saved publication copy.

"""Synthetic split-contract evidence; no estimator training or deployment."""
import hashlib
import json
import platform
import sys
import tempfile
from datetime import datetime, timezone
from pathlib import Path

import numpy as np
import sklearn
from sklearn.model_selection import GroupShuffleSplit, ShuffleSplit

IDS = [f"r{i:02}" for i in range(20)]
UNEQUAL = ["A", "B", "C", "D"] + ["E"] 16
EQUAL = ["A"] 4 + ["B"] 4 + ["C"] 4 + ["D"] 4 + ["E"] 4
MANIFEST = {
    "unequal_sizes": {"A": 1, "B": 1, "C": 1, "D": 1, "E": 16},
    "equal_sizes": {"A": 4, "B": 4, "C": 4, "D": 4, "E": 4},
    "single_group_possible_rows": [1, 1, 1, 1, 16],
    "C1": ["r02"],
    "C2": ["r04", "r05", "r06", "r07", "r08", "r09", "r10", "r11",
           "r12", "r13", "r14", "r15", "r16", "r17", "r18", "r19"],
    "C3": ["r08", "r09", "r10", "r11"],
    "C4": ["r00", "r02"],
    "C5": ["r02"],
    "C6": ["r02", "r05", "r17", "r19"],
}


def require(condition, message):
    if not condition:
        raise RuntimeError(message)


def schema(ids, groups):
    if not ids or len(ids) != len(groups) or len(set(ids)) != len(ids):
        raise ValueError("Missing, unequal-length, or duplicate row identities")
    if any(type(x) is not str or not x for x in ids + groups):
        raise ValueError("Nonempty string row/group identities required")

def audit(ids, groups, train, test, expected_groups=None):
    schema(ids, groups)
    if any(isinstance(i, (bool, np.bool_)) or not isinstance(i, (int, np.integer))
           for i in list(train) + list(test)):
        raise ValueError("Integer split positions required")
    train, test = list(map(int, train)), list(map(int, test))
    n = len(ids)
    if (len(set(train)) != len(train) or len(set(test)) != len(test)
            or set(train) & set(test)
            or set(train) | set(test) != set(range(n))):
        raise ValueError("Split positions must partition the source exactly once")
    train_groups, test_groups = {groups[i] for i in train}, {groups[i] for i in test}
    overlap = sorted(train_groups & test_groups)
    group_count_ok = expected_groups is None or len(test_groups) == expected_groups
    return {
        "train_ids": sorted(ids[i] for i in train),
        "test_ids": sorted(ids[i] for i in test),
        "train_groups": sorted(train_groups), "test_groups": sorted(test_groups),
        "overlap_groups": overlap, "row_count": len(test),
        "group_count": len(test_groups), "source_rows": n,
        "source_groups": len(set(groups)),
        "row_fraction": len(test) / n,
        "group_fraction": len(test_groups) / len(set(groups)),
        "requested_group_count_satisfied": group_count_ok,
        "exact_twenty_percent_rows": len(test) * 5 == n,
        "group_decision": "ACCEPT_GROUP_CONTRACT" if not overlap and group_count_ok else "HOLD",
        "ledger": [{"row_id": ids[i], "group": groups[i],
                    "partition": "test" if i in test else "train"} for i in range(n)],
    }


def main():
    destination = Path(tempfile.mkdtemp(prefix="group-shuffle-", dir=Path(__file__).parent))
    report = {"fixture": "group-shuffle-v1", "status": "FAIL", "cases": [],
              "started_utc": datetime.now(timezone.utc).isoformat(),
              "python": platform.python_version(), "numpy": np.__version__,
              "scikit_learn": sklearn.__version__, "platform": platform.platform(),
              "source_sha256": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
              "manifest": MANIFEST, "input": {"ids": IDS, "unequal": UNEQUAL, "equal": EQUAL}}

    def case(name, function):
        row = {"id": name, "status": "FAIL"}
        report["cases"].append(row)
        function(row)
        row["status"] = "PASS"

    def split_case(row, key, groups, splitter, expected_groups, expected_decision):
        schema(IDS, groups)
        x = np.zeros((len(IDS), 1))
        if isinstance(splitter, GroupShuffleSplit):
            train, test = next(splitter.split(x, groups=np.array(groups)))
        else:
            train, test = next(splitter.split(x))
        row["splitter"] = repr(splitter)
        row["evidence"] = audit(IDS, groups, train, test, expected_groups)
        require(row["evidence"]["test_ids"] == MANIFEST[key], "Unexpected held-out IDs")
        require(row["evidence"]["group_decision"] == expected_decision, "Wrong group decision")

    def rejection(row, operation):
        try:
            operation()
        except ValueError as error:
            row["decision"] = "HOLD"
            row["exception"] = {"type": type(error).__name__, "message": str(error)}
        else:
            raise RuntimeError("Invalid control was accepted")

    try:
        for groups, name in [(UNEQUAL, "unequal_sizes"), (EQUAL, "equal_sizes")]:
            require({g: groups.count(g) for g in sorted(set(groups))} == MANIFEST[name],
                    "Source group inventory differs from literal manifest")
        for key, groups, size, seed, count in [
            ("C1", UNEQUAL, .2, 0, 1), ("C2", UNEQUAL, .2, 5, 1),
            ("C3", EQUAL, .2, 0, 1), ("C4", UNEQUAL, .21, 0, 2),
            ("C5", UNEQUAL, 1, 0, 1)]:
            splitter = GroupShuffleSplit(n_splits=1, test_size=size,
                                         train_size=None, random_state=seed)
            case(key, lambda row, k=key, g=groups, s=splitter, c=count:
                 split_case(row, k, g, s, c, "ACCEPT_GROUP_CONTRACT"))
            if "--inject-failure" in sys.argv and key == "C1":
                raise RuntimeError("Deliberate failure after first recorded case")
        case("C6", lambda row: split_case(row, "C6", UNEQUAL,
             ShuffleSplit(n_splits=1, test_size=.2, random_state=5), None, "HOLD"))
        require(report["cases"][5]["evidence"]["overlap_groups"] == ["E"],
                "Row-sized negative control did not split E")
        case("C7_missing_position", lambda row: rejection(row,
             lambda: audit(IDS, UNEQUAL, list(range(4)), list(range(5, 20)), 1)))
        case("C8_duplicate_position", lambda row: rejection(row,
             lambda: audit(IDS, UNEQUAL, list(range(4)), [4] + list(range(4, 20)), 1)))
        case("C9_duplicate_row_identity", lambda row: rejection(row,
             lambda: schema(["r00"] + IDS[:-1], UNEQUAL)))
        case("C10_missing_group_identity", lambda row: rejection(row,
             lambda: schema(IDS, [""] + UNEQUAL[1:])))
        require(len(report["cases"]) == 10, "Incomplete case inventory")
        require(4 not in MANIFEST["single_group_possible_rows"],
                "One-group fixture unexpectedly permits four rows")
        report["one_group_exact_four_rows_feasible"] = False
        report["status"] = "PASS_10_CASES"
    except Exception as error:
        report["error"] = {"type": type(error).__name__, "message": str(error)}
        raise
    finally:
        report["finished_utc"] = datetime.now(timezone.utc).isoformat()
        path = destination / "results.json"
        path.write_text(json.dumps(report, indent=2, allow_nan=False) + "\n")
        print(path)


if name == "__main__":
    main()

Run the normal qualification and inspect the path printed on standard output:

python group_shuffle_fixture.py

The program creates a unique group-shuffle-* directory beside the source and writes results.json there. The supplied commissioning run exited 0 and reported PASS_10_CASES. Treat that historical record as separate from any later reproduction.

The interruption control is separate:

python group_shuffle_fixture.py --inject-failure
echo $?

The supplied failure run began at 2026-10-12T06:58:49.483238+00:00, ended at 2026-10-12T06:58:49.492813+00:00, exited 1, and retained FAIL, the original input and manifest, completed C1, and a RuntimeError in its own result directory. It tests one deliberate exception after the first recorded case, not every filesystem failure, abrupt process kill, or durability mechanism.

The harness starts both the report and each case at FAIL. A case becomes PASS only after its function completes and all applicable require assertions succeed. The top-level report becomes PASS_10_CASES only after ten case records exist, the feasibility check passes, and no exception escapes. The except block records the current exception; the finally block attempts to persist whatever evidence exists.

C7–C10 are negative controls. Their PASS status means the invalid input was rejected with the expected ValueError; it does not mean the corrupt membership was accepted. The manifest and per-row ledger remain evidence, not decorative output. A missing result file, failed final write, absent required field, or copied result from an older run cannot support a successful new qualification.

5. Inspect a valid group split with too few test rows

C1 uses the unequal fixture, test_size=.2, train_size=None, n_splits=1, and random_state=0. The supplied observation selects group C and test ID r02. Its complementary training IDs are:

r00, r01, r03, r04, r05, r06, r07, r08, r09, r10, r11, r12, r13, r14, r15, r16, r17, r18, r19.

The two size statements are therefore:

• Group allocation: 1 / 5 = 20%

• Row allocation: 1 / 20 = 5%

No group appears in both partitions. The requested one-group count is satisfied. Train and test positions are disjoint, and their union is all twenty source positions. The literal test-ID manifest matches. On that evidence, the case records group_decision="ACCEPT_GROUP_CONTRACT".

The row-size field remains separate: exact_twenty_percent_rows=false. That is not a contradiction and not a scikit-learn defect. It means only that the sentence “.2 held out four of twenty rows” would misstate this API and this result. If the approved requirement was solely to hold out one of five entities, C1 can be accepted. If the approved requirement also demanded exactly four rows, C1 must be held or the report must explain that the row condition was never part of the design.

A percentage without its numerator and denominator would erase this distinction. “20% test set” is insufficient release evidence; “one of five groups, containing one of twenty rows” is reviewable.

6. Hold out the large group without changing the contract

C2 keeps the unequal data and the same group-sized request but changes the declared diagnostic seed to random_state=5. The supplied result selects group E. Its test identities are r04, r05, r06, r07, r08, r09, r10, r11, r12, r13, r14, r15, r16, r17, r18, and r19; the training side is r00, r01, r02, and r03.

Again, exactly one of five groups is held out, so the group fraction is 20%. This time that group contributes sixteen of twenty rows, so the row fraction is 80%. Group E appears only in test; A–D appear only in train. The ledger accounts for all twenty source IDs once. The API contract and the fixture’s complete-group requirement therefore pass even though the exact-four-row diagnostic is false.

That is the headline’s entire claim. It is a bounded counterexample demonstrating that a group fraction does not determine row mass when groups are unequal. It does not establish that 80% is common, desirable, representative, or statistically adequate. Nor should sixteen rows be described as sixteen independent entities: the fixture deliberately says they all belong to one group.

Whether a one-entity test set is useful for a real evaluation remains a design decision. The split audit supplies facts; it does not silently approve the population, sample size, or inference.

Keep counterexample seeds separate from production selection

Seeds 0 and 5 make C1 and C2 reproducible in the pinned commissioning environment. They are not a recommendation to try seeds until the row percentage, class composition, or eventual model score looks attractive.

A fixed seed stabilizes the recorded random choice. It does not repair a poorly defined group population, justify choosing a partition after seeing outcomes, or guarantee identical selected IDs across every future implementation and version. Production selection needs a declared policy and a retained artifact containing actual IDs. The counterexample seeds remain diagnostic evidence, not a seed-shopping procedure.

7. Compare equal groups and a rounded allocation

C3 replaces the unequal mapping with five groups of four rows each. With .2 and seed 0, the supplied test group is C, containing r08, r09, r10, and r11. Now both fractions happen to equal 20%:

• 1 / 5 groups

• 4 / 20 rows

That agreement is real for C3, but it can teach the wrong lesson if it is the only test a maintainer performs. Equal group sizes make the group-count contract look like a row quota. Moving the same code to customers, machines, or sources with unequal contributions then exposes the hidden assumption. A useful qualification suite therefore includes both equal and unequal fixtures.

C4 returns to the unequal mapping and asks for test_size=.21 with seed 0. The documented rounding rule makes the requested count ceil(.21 × 5) = 2 groups. The supplied identities are r00 and r02, belonging to A and C. The evidence must report 2 / 5 = 40% of groups and 2 / 20 = 10% of rows. It must not round the ledger toward a desired “roughly 20% of data” narrative.

C5 uses integer test_size=1 with seed 0. In the pinned commissioning run it reproduces C1: group C and row r02. The important documented meaning is that 1 requests one test group. The repeated identity is a bounded observation of this exact environment, fixture, and seed, not a universal equivalence claim about all random states or future versions.

Together, C3–C5 separate three ideas that are often conflated: equal group mass can make group and row fractions coincide; fractional group requests can round upward; and integer sizes count groups directly. None of those facts eliminates the need to inspect actual membership.

8. Expose the row-sized substitute that crosses an entity

C6 deliberately replaces GroupShuffleSplit with ShuffleSplit(n_splits=1, test_size=.2, random_state=5). The supplied test IDs are r02, r05, r17, and r19. Four of twenty rows meet the fixture’s exact row-size diagnostic.

But group E owns r04 through r19. C6 places r05, r17, and r19 in test while other E rows remain in train. The audit records overlap_groups=["E"] and gives the group decision HOLD.

Nothing mysterious happened. The requirement changed with the splitter. ShuffleSplit selected samples, as its API specifies; it was not asked to keep groups intact. A correct four-row result is therefore insufficient evidence for a new-entity holdout. The sample-sized interface answers “How many rows?” while the grouped interface answers “How many complete groups?” Those questions coincide only under additional data conditions.

Do not “repair” C6 by moving three arbitrary E rows, duplicating rows to make counts work, dropping the oversized entity, or inventing new group labels. Each action changes the declared population or entity definition. A defensible redesign would document the new unit, regenerate the source manifest, rerun membership checks, and obtain approval from the design owner.

The negative comparison is also bounded. This fixture’s row split crosses E; it does not prove every possible row-based split must cross a group. The point is that four selected rows do not establish isolation. When isolation is required, it must be checked directly from identities and group labels rather than inferred from the test length.

9. Reject incomplete or ambiguous membership evidence

C7–C10 test the evidence layer rather than another random split.

C7 supplies train positions 0–3 and test positions 5–19, so source position 4 is missing. C8 supplies train positions 0–3 and test positions beginning with 4 followed by 4–19, duplicating position 4. In both cases, totals or superficial lengths could distract from the defect. audit instead requires unique positions, no train/test intersection, and an exact union equal to set(range(n)); each control raises ValueError and records HOLD.

C9 passes duplicate row identities: ["r00"] + IDS[:-1]. C10 replaces the first group label with an empty string. The schema function rejects both. These checks matter because partition indices only become meaningful when they map to unique, nonempty source identities and explicit groups.

The helper is intentionally narrow. It does not certify every production identifier representation, normalization rule, upstream join, or entity-resolution policy. Within this fixture, however, the declared mapping is authoritative. If production identity quality is uncertain, an accountable data owner must resolve that uncertainty before a split review can claim complete entities.

Missing trustworthy membership evidence has a parallel in historical evaluation: temporal validity needs its own audit. This article does not add timestamps or reconstruct feature availability; it simply makes the same governance principle visible at the row/group boundary.

Preserve errors before interpreting the run status

The --inject-failure run stops after completed C1, records a RuntimeError, retains overall FAIL, and writes the partial record in its own directory according to the supplied observation. It must not be summarized as ten passing cases merely because one case says PASS.

Distinguish three outcomes. An expected local rejection is a passing negative control. An unexpected harness exception makes the run fail even if earlier cases completed. Absence of a required artifact provides no successful qualification at all.

The tested interruption is one Python exception on one path. It does not prove durability under power loss, forced termination, disk exhaustion, or every failed write. Those claims require different controls.

10. Reconcile the split at both levels

The qualification record should let a reviewer see the requested unit, actual identities, containment result, and row-size interpretation without reconstructing them from a percentage.

PASS_10_CASES means all ten diagnostic controls, not ten model evaluations, produced their expected outcomes, including deliberate HOLD classifications. It does not mean all candidate partitions were accepted, and it says nothing about model quality.

The table is a review summary, not a replacement for results.json. Its compact ranges and labels must remain traceable to the full ledger, where each row carries its exact row_id, group, and partition. That detail lets a reviewer reproduce set intersections and source coverage rather than trusting a formatted percentage.

No source total, group total, or selected-row count alone proves that this ledger is complete. The accepted split cases also require exact test IDs, complementary train IDs, zero group overlap, and the literal source mapping. A later run is new evidence and must retain its own identity rather than overwrite the commissioning observation.

11. Prove the bounded feasibility conflict

For the unequal fixture, the complete row counts available from holding out exactly one group are the literal set:

[1, 1, 1, 1, 16]

That set comes directly from A=1, B=1, C=1, D=1, and E=16. The fixture imposes two simultaneous requirements:

• hold out exactly one complete group; and

• hold out exactly four rows.

there is no valid solution in this fixture. One group can contribute either one row or sixteen rows, never four. The code records one_group_exact_four_rows_feasible=False only after confirming that 4 is absent from the manifest’s possible one-group row counts.

The bound must remain precise. This is not a theorem that every group-disjoint partition of the twenty rows must miss four. Holding out four of the small groups would produce exactly four test rows, but it would hold out 4 / 5 = 80% of groups. That is a different design and a different entity-coverage claim.

The evidence therefore exposes a requirements conflict; it does not authorize an optimization search, a seed sweep, or a silent change from one group to four. The accountable owner may revise the group-count policy, replace the exact row quota with an approved range, redefine the eligible population, or decide that this dataset cannot support the desired holdout. Each option changes the contract and requires new evidence.

The review should stop before fitting anything. No estimator, metric, or favorable score can make an impossible membership combination feasible. Likewise, manually assembling four rows would meet the row count only by abandoning the complete-group requirement unless the approved design itself changes.

12. Decide whether to accept mechanics, repair a claim or redesign

The release decision should name the failing contract and the owner empowered to address it.

A HOLD is an operational classification for the split, evidence, or design; it is not a reason to suppress an explanatory article. An approved range must precede inspection of candidate splits. Declaring “5% is close enough” after C1, or “80% is acceptable” after C2, converts evidence into post-hoc policy. The same rule applies to changing the number of test groups after observing which entities were drawn.

Separate this evidence from statistical and temporal validity

The scikit-learn 1.8 cross-validation guide explains why grouped observations can require holding out group subsets and notes that successive randomized GroupShuffleSplit partitions have no guaranteed relationship. This article uses one split per case, so it is neither an exhaustive group partition nor evidence that every entity receives test coverage exactly once.

Correct membership mechanics also do not prove representativeness, adequate statistical power, class balance, effective sample size, or valid inference. Those are sampling assumptions beyond row counts. Temporal validity remains the separate boundary identified earlier: complete group identities do not establish that features or labels were available at the required historical instant.

No score is calculated here. The fixture does not evaluate performance, calibration, thresholds, preprocessing, or feature leakage. Its narrow contribution is to prove what was selected, in which unit, and whether that membership satisfies the declared split contract.

13. Repair split artifacts and assign review ownership

A defensible split artifact preserves the source snapshot or immutable reference, the row-to-group map, parameter values and units, n_splits, train_size, seed, library versions, actual train/test indices, test IDs, group sets, overlap result, and both fractions. The source hash and result directory make the executable evidence traceable. These are practical scikit-learn foundations for reproducible work, not a substitute for design approval.

When only the prose is wrong, repair the claim while retaining the original split and its evidence. For example, change “20% of rows were held out” to “one of five groups was held out, containing sixteen of twenty rows.” Do not regenerate a more convenient partition merely to preserve the old wording.

When membership violates the approved design, create a new reviewed split artifact. Mark downstream analyses, fitted objects, reports, and decisions that depended on the rejected membership for reevaluation. Relabeling “20%” as “80%” cannot repair an old model score, because this laboratory never establishes whether that score was valid.

Ownership should be explicit:

• The ML maintainer implements partition generation and evidence capture.

• The evaluation owner approves the sampling unit, acceptable group/row conditions, and release decision.

• The report owner states the observed numerators and denominators accurately.

Store the report as an append-only review input where practical. A replacement artifact should identify the superseded record and why a new split was authorized; it should not erase the evidence that informed an earlier decision. This is especially important after anyone has seen downstream results, because apparently harmless regeneration can become an unrecorded test-set selection step.

Requalify when the source population changes, entities are regrouped, rows are added or removed, splitter settings change, or relevant runtime versions change. A retained seed is not enough if its inputs or implementation context differ. Preserve old artifacts rather than silently replacing them after outcomes have influenced review.

14. Build the foundations for a defensible split review

On the programme page checked on October 12, 2026, Refonte Learning’s Data Science & AI programme presents a three-month path requiring 12–14 hours per week covering Python Data Science, statistical modelling, machine learning and predictive modelling, problem-solving, and applied projects. Its FAQ names Python, Jupyter Notebook, pandas, NumPy, Matplotlib, scikit-learn, and TensorFlow; the page also describes a Training Certificate and Certificate of Internship on successful completion. The page does not establish that this exact GroupShuffleSplit laboratory or these pinned versions are taught, so advanced readers should check the current syllabus directly.

The concrete habit to carry into any review is simple: record the held-out groups and every row they own before describing the size of the test set.