The Optimizer Loaded. Which Parameters Received Its State?

A training maintainer changes how an optimizer is constructed, restores the model, restores the optimizer, and gets no exception. The named model values are exactly where they belong. The optimizer's load_state_dict() call also returns normally. Yet the first controlled update after restoration moves the named parameters to different values than the approved continuation.

That is the scenario tested here. It is deliberately smaller than a real training stack: one CPU process, PyTorch 2.10.0+cpu, float64, two same-shaped scalar parameters, one SGD parameter group, deterministic gradients, and locally generated in-memory state dictionaries. The experiment does not claim a production incident or an industry failure rate.

The important distinction is between two restoration contracts. Module.load_state_dict() copies model parameters and buffers according to state-dictionary keys, with strict=True requiring an exact key match. Optimizer state has a different documented correspondence rule: saved parameter identifiers are associated with the current optimizer's actual Parameter objects by group position, without an additional semantic identity check at that matching step.

That means a successful PyTorch optimizer load is not, by itself, proof that left recovered left's momentum. The proof has to reconcile the source order, current live objects, semantic model names, loaded metadata, saved momentum, and a controlled numerical update.

The decision at the end is intentionally bounded. A reviewer can accept the demonstrated compatible restoration, hold an incompatible candidate, rebuild the optimizer in a verified saved order while preserving momentum, or deliberately start a different phase with fresh optimizer state. Those are four different decisions.

Define the parameter-to-state contract and freeze the baseline

Start with the contract that the continuation is supposed to preserve. The approved source in this experiment is one paired model-and-optimizer capture taken after one controlled SGD step. It contains four logically separate facts:

Do not collapse those into “the checkpoint loaded.” PyTorch documents that an optimizer state_dict contains per-parameter optimization state plus parameter groups. The saved params entries are identifiers associating state with the group; the parameter objects themselves are not saved. During restoration, PyTorch documents matching by zipping the saved identifiers with the current group's actual Parameter objects in order, “without additional verification.”

That wording should be read narrowly. It does not mean the optimizer API performs no validation whatsoever. The operational issue here is the absence of semantic identity verification at that positional correspondence boundary. The fixture deliberately leaves the supported topology structurally compatible so that the semantic-order question is isolated.

PyTorch's optimizer overview separately warns that parameter collections need deterministic ordering that is consistent between runs. A list can satisfy deterministic iteration on every individual execution and still change from [left, right] to [right, left] after an optimizer-construction refactor.

Refonte Learning's earlier retained-SGD frozen-parameter audit already makes the positional-association warning while studying a different question: whether a retained optimizer can move a nominally frozen parameter under stale, zero, or absent gradient state. This article does not repeat that freeze experiment. Here every parameter remains trainable; the variable is restoration order.

The source model begins with:

left  = 10
right = 20

objective = 2*left + 8*right

Therefore the controlled gradients are exactly:

d objective / d left  = 2
d objective / d right = 8

SGD is fixed to:

lr             = 0.125
momentum       = 0.5
dampening      = 0
weight_decay   = 0
nesterov       = False
maximize       = False
foreach        = False
differentiable = False
fused          = False

PyTorch 2.10 documents both the momentum recurrence and its first-step initialization: with momentum enabled, the initial momentum buffer is the first gradient rather than an all-zero buffer; on subsequent steps the momentum update uses the previous buffer plus the new gradient according to the configured momentum and dampening.

So after the source's first step:

left buffer  = 2
left weight  = 10 - 0.125*2 = 9.75

right buffer = 8
right weight = 20 - 0.125*8 = 19

Both parameters have shape [1], dtype torch.float64, and device cpu. Making them structurally identical is deliberate: a shape or dtype comparison cannot accidentally reveal that their optimizer state was exchanged.

The capture-time manifest therefore records the actual ordered semantic names derived from the source model's live Parameter objects, plus shape/dtype/device signatures, source identifier synthetic-pair-step-1, and step 1. PyTorch's named_parameters() API is the appropriate model-side primitive because it yields each parameter's semantic name together with the actual Parameter object.

The key word is capture-time. A supposedly independent manifest created after restoration by reading newly loaded param_names would be circular evidence. The fixture instead derives ownership from the source model's real objects before any candidate restoration is attempted.

The manifest is also not an authentication system. Its provenance is accepted as part of this local fixture's contract. In a larger system, source authenticity and artifact integrity would be separate concerns. Here the policy is simpler: stable semantic names plus a trusted original ordered-name manifest are prerequisites for compatible reconstruction. If that correspondence evidence is absent or contradictory, the decision is HOLD.

The supported outcomes are:

ACCEPT when the live order and independently expected named state already agree; HOLD when correspondence evidence is missing or incompatible; REPAIR THEN ACCEPT when the topology is unchanged and the current optimizer can be reconstructed in the verified saved order; and SEPARATE RESTART POLICY when the owner intentionally abandons old optimizer state and begins with fresh momentum.

Run the complete optimizer-order fixture

The bounded preparation run supplied for this article was recorded on October 26, 2026 at 06:11:55.368373 UTC using CPython 3.12.14, PyTorch 2.10.0+cpu, PyTorch Git identifier 449b1768410104d3ed79d3bcfe4ba1d65c7f22c0, Linux-6.18.55-x86_64-with-glibc2.39, no CUDA build, CPU float64, one PyTorch thread, and deterministic algorithms enabled. The supplied source fingerprint is:

8777c4e5d41e2323b49d63883d2667261114b41013cf7fb6d3bd984c03208fcc

That fingerprint describes the supplied preparation source. A modified byte sequence is a different fixture and needs a separately recorded run; it should not inherit this run's environment or result record.

Prepare the isolated environment with the pinned CPU wheel:

python3.12 -m venv .venv
.venv/bin/python -m pip install --index-url https://download.pytorch.org/whl/cpu 'torch==2.10.0+cpu'
.venv/bin/python optimizer_order_fixture.py > optimizer_order_evidence.json

The complete commissioning fixture is:

import copy
import hashlib
import json
import platform
import sys
from datetime import datetime, timezone
from pathlib import Path

import torch

NAMES = ["left", "right"]
OPTIONS = dict(lr=0.125, momentum=0.5, dampening=0.0, weight_decay=0.0,
              nesterov=False, maximize=False, foreach=False,
              differentiable=False, fused=False)
SAVED_WEIGHTS = {"left": 9.75, "right": 19.0}
SAVED_MOMENTUM = {"left": 2.0, "right": 8.0}
RESUMED_WEIGHTS = {"left": 9.375, "right": 17.5}
RESUMED_MOMENTUM = {"left": 3.0, "right": 12.0}


def require(condition, message):
    if not condition:
        raise RuntimeError(message)


class Pair(torch.nn.Module):
    def init(self):
        super().__init__()
        self.left = torch.nn.Parameter(torch.tensor([10.0], dtype=torch.float64, device="cpu"))
        self.right = torch.nn.Parameter(torch.tensor([20.0], dtype=torch.float64, device="cpu"))

    def forward(self):
        return 2.0 self.left + 8.0 self.right


def optimizer_for(model, order, lr=0.125):
    named = dict(model.named_parameters())
    group = {"params": [named[name] for name in order], "param_names": list(order)}
    return torch.optim.SGD([group], **(OPTIONS | {"lr": lr}))


def actual_order(model, optimizer):
    names_by_id = {id(p): name for name, p in model.named_parameters()}
    require(len(optimizer.param_groups) == 1, "Only one group is supported")
    return [names_by_id[id(p)] for p in optimizer.param_groups[0]["params"]]



def weights(model):
    return {name: p.detach().item() for name, p in model.named_parameters()}


def momentum(model, optimizer):
    result = {}
    for name, p in model.named_parameters():
        value = optimizer.state.get(p, {}).get("momentum_buffer")
        result[name] = None if value is None else value.detach().item()
    return result


def observe(model, optimizer):
    return {"actual_parameter_order": actual_order(model, optimizer),
            "loaded_param_names": list(optimizer.param_groups[0]["param_names"]),
            "weights": weights(model), "momentum_by_actual_name": momentum(model, optimizer),
            "learning_rate": optimizer.param_groups[0]["lr"]}


def step(model, optimizer):
    optimizer.zero_grad(set_to_none=True)
    model().sum().backward()
    require({n: p.grad.item() for n, p in model.named_parameters()}
            == {"left": 2.0, "right": 8.0}, "Unexpected controlled gradients")
    optimizer.step()


def signature(model):
    return {n: {"shape": list(p.shape), "dtype": str(p.dtype), "device": str(p.device)}
            for n, p in model.named_parameters()}


def source_bundle():
    model = Pair()
    optimizer = optimizer_for(model, NAMES)
    step(model, optimizer)
    require(weights(model) == SAVED_WEIGHTS, "Wrong first-step weights")
    require(momentum(model, optimizer) == SAVED_MOMENTUM, "Wrong first-step momentum")
    return {"model": copy.deepcopy(model.state_dict()),
            "optimizer": copy.deepcopy(optimizer.state_dict()),
            "manifest": {"source_id": "synthetic-pair-step-1", "step": 1,
                         "ordered_names": actual_order(model, optimizer),
                         "signature": signature(model)}}


def fresh_model(bundle):
    model = Pair()
    loaded = model.load_state_dict(copy.deepcopy(bundle["model"]), strict=True, assign=False)
    require(loaded.missing_keys == [] and loaded.unexpected_keys == [], "Bad model restore")
    require(weights(model) == SAVED_WEIGHTS, "Model values not restored by name")
    return model


def validate_manifest(model, bundle):
    manifest = bundle.get("manifest")
    require(isinstance(manifest, dict), "HOLD: missing manifest")
    names = manifest.get("ordered_names")
    require(isinstance(names, list) and all(isinstance(n, str) for n in names),
            "HOLD: malformed names")
    require(len(names) == len(set(names)), "HOLD: duplicate names")
    require(set(names) == set(dict(model.named_parameters())), "HOLD: name inventory mismatch")
    require(manifest.get("signature") == signature(model), "HOLD: signature mismatch")
    groups = bundle["optimizer"]["param_groups"]
    require(len(groups) == 1 and len(groups[0]["params"]) == len(names),
            "HOLD: group topology mismatch")
    require(groups[0].get("param_names") == names, "HOLD: source metadata mismatch")
    require(all(groups[0].get(k) == v for k, v in OPTIONS.items()),
            "HOLD: optimizer options mismatch")
    return names


def checked_load(model, optimizer, bundle):
    names = validate_manifest(model, bundle)
    require(actual_order(model, optimizer) == names, "HOLD: parameter order mismatch")
    optimizer.load_state_dict(copy.deepcopy(bundle["optimizer"]))
    require(momentum(model, optimizer) == SAVED_MOMENTUM, "HOLD: wrong named momentum")


def run(report):
    require(torch.__version__ == "2.10.0+cpu", "Use torch 2.10.0+cpu")
    require(torch.version.cuda is None, "Use the CPU build")
    torch.set_num_threads(1)
    torch.use_deterministic_algorithms(True)
    report["environment"]["threads"] = torch.get_num_threads()
    report["environment"]["deterministic_algorithms"] = torch.are_deterministic_algorithms_enabled()
    bundle = source_bundle()
    rows = report["cases"]
    rows.append({"case": "source_after_one_step", "decision": "BASELINE",
                 "weights": SAVED_WEIGHTS, "momentum": SAVED_MOMENTUM,
                 "manifest": copy.deepcopy(bundle["manifest"]),
                 "serialized_parameter_ids": bundle["optimizer"]["param_groups"][0]["params"]})

    model = fresh_model(bundle)
    optimizer = optimizer_for(model, NAMES, lr=0.5)
    checked_load(model, optimizer, bundle)
    before = observe(model, optimizer)
    require(before["learning_rate"] == 0.125, "Saved learning rate was not restored")
    step(model, optimizer)
    require(weights(model) == RESUMED_WEIGHTS, "Wrong correct-order resumed weights")
    require(momentum(model, optimizer) == RESUMED_MOMENTUM, "Wrong resumed momentum")
    rows.append({"case": "same_order", "decision": "ACCEPT_BOUNDED_RESUME",
                 "constructor_lr": 0.5, "before_step": before,
                 "after_step": observe(model, optimizer)})

    model = fresh_model(bundle)
    optimizer = optimizer_for(model, list(reversed(NAMES)))
    returned = optimizer.load_state_dict(copy.deepcopy(bundle["optimizer"]))
    before = observe(model, optimizer)
    require(returned is None, "Unexpected optimizer load return")
    require(before["actual_parameter_order"] == ["right", "left"], "Order changed")
    require(before["loaded_param_names"] == NAMES, "Saved names did not overwrite metadata")
    require(before["weights"] == SAVED_WEIGHTS, "Optimizer load altered weights")
    require(before["momentum_by_actual_name"] == {"left": 8.0, "right": 2.0},
            "Wrong reordered association")
    step(model, optimizer)
    require(weights(model) == {"left": 9.0, "right": 17.875}, "Wrong reordered update")
    require(momentum(model, optimizer) == {"left": 6.0, "right": 9.0}, "Wrong reordered momentum")
    rows.append({"case": "reordered_successful_load", "decision": "REJECT_RESUME",
                 "load_return": returned, "before_step": before,
                 "after_step": observe(model, optimizer)})


    mutations = {
        "reordered_gate": lambda b: None,
        "missing_manifest": lambda b: b.pop("manifest"),
        "duplicate_names": lambda b: b["manifest"].update(ordered_names=["left", "left"]),
        "unknown_name": lambda b: b["manifest"].update(ordered_names=["left", "renamed"]),
        "signature_mismatch": lambda b: b["manifest"]["signature"]["right"].update(shape=[2]),
        "source_metadata_mismatch": lambda b: b["optimizer"]["param_groups"][0].update(
            param_names=["right", "left"]),
    }
    expected_errors = {
        "reordered_gate": "HOLD: parameter order mismatch",
        "missing_manifest": "HOLD: missing manifest",
        "duplicate_names": "HOLD: duplicate names",
        "unknown_name": "HOLD: name inventory mismatch",
        "signature_mismatch": "HOLD: signature mismatch",
        "source_metadata_mismatch": "HOLD: source metadata mismatch",
    }
    for case, mutate in mutations.items():
        candidate = copy.deepcopy(bundle)
        mutate(candidate)
        model = fresh_model(bundle)
        order = list(reversed(NAMES)) if case == "reordered_gate" else NAMES
        optimizer = optimizer_for(model, order)
        before = observe(model, optimizer)
        try:
            checked_load(model, optimizer, candidate)
        except RuntimeError as exc:
            require(str(exc) == expected_errors[case], "Wrong rejection: " + str(exc))
            reason = str(exc)
        else:
            raise RuntimeError(case + ": negative control passed")
        after = observe(model, optimizer)
        require(after == before, case + ": rejection mutated the candidate")
        rows.append({"case": case, "decision": "HOLD", "reason": reason,
                     "before": before, "after": after, "unchanged": True})

    model = fresh_model(bundle)
    order = validate_manifest(model, bundle)
    optimizer = optimizer_for(model, order)
    checked_load(model, optimizer, bundle)
    step(model, optimizer)
    require(weights(model) == RESUMED_WEIGHTS, "Reconstruction did not restore this trajectory")
    require(momentum(model, optimizer) == RESUMED_MOMENTUM, "Reconstruction lost state")
    rows.append({"case": "manifest_order_reconstruction", "decision": "REPAIR_THEN_ACCEPT",
                 "after_step": observe(model, optimizer)})

    model = fresh_model(bundle)
    optimizer = optimizer_for(model, NAMES)
    require(momentum(model, optimizer) == {"left": None, "right": None}, "Not fresh")
    step(model, optimizer)
    require(weights(model) == {"left": 9.5, "right": 18.0}, "Wrong reset trajectory")
    require(weights(model) != RESUMED_WEIGHTS, "Reset accidentally equalled resume")
    rows.append({"case": "explicit_optimizer_reset", "decision": "SEPARATE_RESTART_POLICY",
                 "after_step": observe(model, optimizer)})
    require(len(rows) == 11, "Wrong case count")
    report["status"] = "PASS"


if name == "__main__":
    report = {"started_utc": datetime.now(timezone.utc).isoformat(), "status": "FAIL",
              "environment": {"python": sys.version, "python_implementation": platform.python_implementation(),
                              "platform": platform.platform(), "torch": str(torch.__version__),
                              "torch_git": torch.version.git_version, "cuda_build": torch.version.cuda,
                              "torch_config": torch.__config__.show(), "device": "cpu", "dtype": "torch.float64",
                              "fixture_sha256": hashlib.sha256(Path(__file__).read_bytes()).hexdigest()},
              "cases": []}
    try:
        run(report)
    except Exception as exc:
        report["failure"] = {"type": type(exc).__name__, "message": str(exc)}
    print(json.dumps(report, indent=2, allow_nan=False))
    sys.exit(0 if report["status"] == "PASS" else 1)

The harness is fail-closed with respect to its declared assertions. It initializes status to FAIL, records an exception if setup or an expectation breaks, and exits nonzero unless all expected cases finish and run() sets status to PASS. A missing PyTorch installation, wrong wheel, unexpected arithmetic result, or malformed environment therefore does not count as successful detection of the optimizer-order problem.

The supplied preparation ledger contains eleven cases:

A harness PASS means those positive controls and intentionally rejected controls behaved as asserted in this bounded preparation run. It is not a statement that every incoming optimizer bundle is safe.

Separate successful model restoration from optimizer correspondence

Before testing optimizer order, prove that the model itself restored correctly.

fresh_model() creates a new Pair, then calls:

model.load_state_dict(
    copy.deepcopy(bundle["model"]),
    strict=True,
    assign=False,
)

PyTorch 2.10 documents Module.load_state_dict() as copying parameters and persistent buffers from the supplied state dictionary into the module. With strict=True, the incoming keys must exactly match the module's state-dictionary keys; the result exposes missing_keys and unexpected_keys. With assign=False, the receiving module's tensor properties are preserved rather than replacing them according to the loaded tensors' properties.

The fixture checks both returned lists are empty and independently checks the named values:

left  = 9.75
right = 19.0

That establishes what this experiment needs from model restoration. It deliberately does not reproduce Refonte Learning's separate model receiver checks after load_state_dict errors, which examine failed model loads and possible receiver mutation. Here model restoration succeeds cleanly; the question begins afterward, when an optimizer is constructed around the restored live parameters.

The positive optimizer control uses the same semantic order as the source:

source order:  [left, right]
current order: [left, right]

It intentionally constructs that optimizer with lr=0.5, then loads the source optimizer state. The supplied observation shows the loaded group has lr=0.125, demonstrating that the saved optimizer configuration, not the constructor control value, is the one used for this restored step.

Now change exactly one thing: construct the current optimizer in reverse order.

source order:  [left, right]
current order: [right, left]

The live model still contains the correct named weights. Both parameters still have shape [1], dtype float64, and CPU placement. The optimizer still has one group containing two parameters.

The direct call:

returned = optimizer.load_state_dict(copy.deepcopy(bundle["optimizer"]))

returns None normally in the supplied run. Yet the state ledger immediately afterward is:

That is the core PyTorch optimizer state mismatch.

The optimizer-state identifiers should not be interpreted as durable model names. PyTorch's 2.10 Optimizer.state_dict reference says the state mapping uses parameter IDs, the parameter itself is not saved, and the group contains lists of those IDs. It explicitly says they may look like indices but are association identifiers, and restoration matches them against actual current parameters by order.

The fixture's actual_order() therefore does something stronger than printing optimizer metadata:

names_by_id = {id(p): name for name, p in model.named_parameters()}
return [names_by_id[id(p)] for p in optimizer.param_groups[0]["params"]]

id(p) is only process-local instrumentation here. It is used to join two live views of the same objects: the model's named_parameters() mapping and the optimizer's actual params list. It is not proposed as a persistent checkpoint identity.

Likewise, momentum() does not zip a metadata name list with state slots. It walks the model's actual (name, Parameter) pairs and asks optimizer.state[p] for that specific live object's buffer.

That produces the semantic evidence that matters:

actual current order  = [right, left]
model weights         = {left: 9.75, right: 19.0}
momentum by real name = {left: 8.0, right: 2.0}

A useful restoration ledger should keep these concepts in separate columns:

The equal shapes are intentionally unhelpful. The distinct starting weights are also not used as identities. The join is made from the actual model objects to their semantic names.

Treat param_names as metadata, then prove the update mathematically

The negative control becomes particularly dangerous if a reviewer inspects only param_names.

PyTorch's optimizer loading documentation is explicit: parameter names in a loaded state's param_names entries do not affect the loading process. When those names are present in the incoming state, they are saved into the optimizer and override the current names, if present. PyTorch documents a custom load-state pre-hook as the mechanism for cases that intentionally need name-based adaptation; implementing such a migration is outside this experiment.

That contract explains the apparently reassuring observation:

before load:
    actual objects     = [right, left]
    current metadata   = [right, left]

after load:
    actual objects     = [right, left]
    loaded param_names = [left, right]

The labels changed. The objects did not.

If an audit printed only:

optimizer.param_groups[0]["param_names"]

it would see:

["left", "right"]

and could wrongly conclude that the saved left state was remapped to the live left parameter. PyTorch explicitly says that is not what those names do during ordinary loading.

The three evidence sources must therefore remain separate:

Names are still valuable. The fixture requires the saved metadata to agree with the independent source manifest. What names cannot do is convert an ordinary load_state_dict() call into semantic name matching.

Now make the consequence independently calculable.

For this configuration, PyTorch's documented momentum form reduces to:

and:

because dampening and weight decay are zero, Nesterov and maximize are disabled, and the momentum buffer already exists.

For the correct restoration, left starts the resumed step with buffer 2 and gradient 2:

right starts with buffer 8 and gradient 8:

So the approved one-step continuation oracle is:

momentum: left=3, right=12
weights:  left=9.375, right=17.5

For the reversed association, the real left parameter inherited the saved buffer 8:

And the real right parameter inherited saved buffer 2:

The failing ledger is therefore:

Nothing about those numbers requires a quality judgment. The synthetic expression is not a loss benchmark, and the experiment contains no dataset or model-quality metric. The different next update is evidence that the optimizer state was associated with different semantic parameters.

This fixture can use exact Python-float equality because every relevant literal (integers, 0.5, 0.125, and the derived quarter/eighth increments) is exactly representable in binary floating point in this path. That is a fixture-specific convenience. It is not guidance to use zero tolerance for arbitrary neural-network numerics.

Put the HOLD gate before optimizer loading, then reconstruct the compatible order

A practical review should detect the incompatible candidate before loading optimizer state and before taking an optimizer step.

The fixture divides that job between validate_manifest() and checked_load().

Operationally, the order is:

1. Require the independent manifest to exist.

2. Require a valid list of unique semantic names.

3. Require the names to equal the current model's complete parameter-name inventory.

4. Require the saved shape/dtype/device signature to equal the fresh restored model.

5. Require the supported one-group topology and expected parameter count.

6. Require incoming param_names metadata to agree with the independent source order.

7. Reconcile the saved optimizer options.

8. Resolve the optimizer's actual live parameter order through object identity and require it to equal the source manifest.

9. Only then call optimizer load_state_dict().

10. After loading, compare momentum by actual semantic model name to the fixture's independent expected momentum.

The final momentum comparison matters. Preflight structure is evidence about compatibility; it is not authentication of arbitrary state contents. In this tiny fixture, the literal expected map:

{"left": 2.0, "right": 8.0}

provides an independent post-load oracle. A real model would need its own retained expected correspondence and state evidence. Copying these numeric thresholds into another model would prove nothing.

The six supplied negative controls all stop before optimizer-state loading and before the next optimizer step:

For all six, the supplied run checks that before == after. The fresh candidate therefore remains at restored model weights:

left  = 9.75
right = 19

and still has no loaded momentum state. Rejection itself has not mutated the candidate.

This is why negative-control quality matters. A test suite where every case throws because torch is missing has not demonstrated a useful association gate. Nor is “some exception printed” sufficient. The loop catches the intended RuntimeError, verifies its message equals the exact expected reason, checks candidate invariance, and fails the entire harness if a negative control unexpectedly passes.

The constructive repair starts from another fresh correctly restored model, not from the rejected object after an incorrect update.

The sequence is:

model = fresh_model(bundle)
order = validate_manifest(model, bundle)
optimizer = optimizer_for(model, order)
checked_load(model, optimizer, bundle)
step(model, optimizer)

The trusted manifest says:

ordered_names = ["left", "right"]

Those names are resolved through the current model's named_parameters() mapping. That yields the current live left and right objects in the approved source order. The new optimizer is constructed around those objects, and an independent copy of the original optimizer state is then loaded.

The supplied repair result is:

after next step:
weights  = {left: 9.375, right: 17.5}
momentum = {left: 3.0, right: 12.0}

That equals the positive same-order control.

This is a bounded reconstruction, not a generic migration mechanism. It preserves the exact supported topology: the same two semantic parameters, the same one group, the same signatures, and the same optimizer semantics. Do not “repair” a mismatch by sorting arbitrary parameter names alphabetically, rewriting param_names until the labels appear aligned, or assigning state according to shape. Those techniques manufacture agreement rather than demonstrate correspondence.

A renamed parameter, an added or removed parameter, a changed grouping scheme, or another optimizer topology belongs to a separate migration review. PyTorch documents load-state pre-hooks for intentional state-dictionary adaptation, but that facility is deliberately excluded here.

When the old correspondence cannot be independently recovered, HOLD is the conservative result.

Keep reconstruction, optimizer reset, and post-error recovery as different decisions

Reconstructing the source order is not equivalent to resetting the optimizer.

The fixture's reset control starts from the same correctly restored model values:

left  = 9.75
right = 19

but constructs a fresh optimizer and deliberately does not load the source optimizer state.

Before the step:

momentum(left)  = None
momentum(right) = None

PyTorch's SGD documentation states that the first momentum buffer is initialized from that step's gradient.

Therefore:

left:
    first buffer = 2
    new weight   = 9.75 - 0.125*2
                 = 9.5

right:
    first buffer = 8
    new weight   = 19 - 0.125*8
                 = 18

The reset path produces:

weights  = {left: 9.5, right: 18.0}
momentum = {left: 2.0, right: 8.0}

Compare all three possibilities:

Silently deleting momentum because correspondence is uncertain does not recover the demonstrated continuation. It selects a different training-state policy.

That may be a legitimate policy. A training owner can explicitly decide that a refactor marks the beginning of a new optimization phase. The operational requirement is to call it what it is and record the boundary. “Fresh optimizer by design” and “restored prior optimizer state” should never be represented as interchangeable evidence.

There is a second boundary: what to do if the incorrect resumed step has already run.

Suppose the reversed optimizer took the demonstrated step. The model now contains:

left  = 9.0
right = 17.875

Its optimizer state now contains:

left momentum  = 6
right momentum = 9

Swapping two buffers afterward does not undo the parameter update that was computed from those wrong buffers. Rewriting param_names also does not return the model to 9.75/19.

Recovery in this fixture means discarding that advanced pair and recreating both model and optimizer from the verified pre-step source:

1. Restore a fresh model from the approved source model state.

2. Revalidate the source manifest.

3. Resolve the approved ordered names to the new live parameter objects.

4. Construct the optimizer in that order.

5. Load the untouched source optimizer state.

6. Re-run the controlled verification step.

That requirement makes upstream source preservation important. Refonte Learning's article on preserving an independently selected model snapshot treats that earlier capture problem separately; this experiment assumes its source model and optimizer dictionaries are already independently owned copies.

If the verified paired pre-step source no longer exists, do not pretend that metadata surgery recovers it. The decision is HOLD, or an explicitly governed choice to begin a new run or phase with newly initialized optimizer state.

The evidence to retain after a detected incorrect step is correspondingly specific: the approved source bundle, model/optimizer pairing identifier, exact completed step boundary, runtime and optimizer configuration, fixture fingerprint, actual-vs-intended ownership ledger, and the recovery source used for reconstruction.

Apply bounded acceptance criteria instead of declaring an exact resume

For this experiment, acceptance is a multi-part correspondence decision, not the absence of an exception.

An ACCEPT result for the supplied fixture requires all of the following to agree:

model named values:
    left  = 9.75
    right = 19

source and actual optimizer order:
    [left, right]

configuration:
    lr=0.125
    momentum=0.5
    dampening=0
    weight_decay=0
    nesterov=False
    maximize=False
    foreach=False
    differentiable=False
    fused=False

loaded named momentum:
    left  = 2
    right = 8

controlled next step:
    left  = 9.375
    right = 17.5

next momentum:
    left  = 3
    right = 12

negative controls:
    all six reject with the intended exact reason
    all six leave their candidates unchanged

The model-side checks and optimizer-side checks answer different questions. Module.load_state_dict(strict=True, assign=False) proves that, within this simple model contract, the expected model keys were accepted and the independently checked named values were restored. Optimizer.load_state_dict() then restores a different class of state whose documented parameter correspondence is positional, not semantic-name based.

A prediction from the restored model would add no missing optimizer evidence here. Before the resumed optimizer step, both the correct-order and wrong-order cases contain exactly the same model weights, so any forward computation that depends only on those weights is identical. The divergence appears when the saved state participates in the next update.

The scope stops there. This fixture does not restore or validate a learning-rate scheduler, stochastic-number state, data position, data-loader state, distributed partitions, mixed-precision machinery, or any other training subsystem. It does not deserialize a checkpoint file. It contains no model-quality evaluation. It qualifies one model/SGD state restoration and one subsequent controlled update.

Likewise, training-state acceptance should not be substituted for serving-state acceptance. Refonte Learning's separate PyTorch serving-mode validation examines module training/evaluation mode and gradient context as a different boundary.

The appropriate statement after this fixture passes is therefore:

For this pinned PyTorch 2.10.0+cpu, one-group, CPU-float64 SGD experiment, the current semantic parameters received the independently expected saved momentum and produced the independently predicted next update.

It is not:

Exact full-training continuation is guaranteed.

That distinction is especially important when interpreting the ordered manifest. The manifest lets this policy reconstruct the supported original topology. It does not prove that arbitrary incoming bytes are authentic, and the fixed 2/8 momentum oracle is meaningful only for this fixture. A real training system needs independently retained expected correspondence evidence appropriate to its own parameter set.

Assign ownership, evidence retention, and the learning boundary

A dependable optimizer restore needs an owner for each side of the contract.

The training owner owns the paired source: which model state and optimizer state belong to the same training boundary, what optimizer semantics are intended, whether momentum is supposed to continue, and whether a fresh optimizer represents an intentional phase transition.

The code-change owner owns current construction. A refactor that changes a list comprehension, name filter, module traversal, or explicit optimizer group can preserve every model name and shape while changing the order of actual Parameter objects presented to the optimizer. PyTorch's requirement is deterministic order that remains consistent across runs; merely producing a deterministic list in both code versions does not prove those two lists are identically ordered.

The reviewer owns independent reconciliation. The review ledger should retain at least:

PyTorch version/build
Python/runtime identity
source identifier
source step
model named values
ordered semantic parameter manifest
shape/dtype/device signatures
optimizer options
actual current live parameter order
loaded param_names metadata
momentum by actual semantic name
fixture/source fingerprint
negative-control outcomes
controlled next-step oracle and observation
recovery source
final decision

Refonte Learning's broader discussion of model deployment and versioning practices treats model/version traceability in the wider deployment lifecycle; this article's manifest is the narrower training-state evidence required at this particular optimizer handoff.

Revalidation is warranted when any premise of the accepted contract changes: parameter order, parameter inventory, group topology, optimizer semantics, relevant runtime behavior, or the verification fixture itself. The PyTorch documentation references used here are deliberately the versioned 2.10 pages accessed for the October 26, 2026 research cutoff; they are not presented as evidence about an unspecified newer runtime. The versioned torch.optim page itself displays creation on June 13, 2025 and update on August 24, 2025, but those page dates do not establish when the documented association behavior was introduced.

The primary operational principle is simple: never use just-loaded optimizer labels as the evidence that justified the load. Build the expected correspondence from an independent source, join current semantic names to the actual live Parameter objects, inspect state through those objects, and make the next controlled update calculable.

For engineers developing that kind of model-state and verification discipline, Refonte Learning's AI Engineering program is currently described as a three-month program requiring about 12–14 hours per week. Its published scope includes neural networks and model training, AI model development and optimization, scaling AI systems, and practical/capstone work; its FAQ explicitly names PyTorch, TensorFlow, and Keras among the tools covered. The page does not establish that this exact optimizer-restoration laboratory or this PyTorch version is part of the curriculum, so neither should be implied.

The acceptance rule remains independent of any course or credential: restore only when model values, trusted source order, actual live optimizer order, configuration, named momentum, and the controlled next update all reconcile. When the topology is still compatible but construction order changed, rebuild the optimizer in the verified saved order and preserve its momentum. When that correspondence cannot be established, HOLD. When the owner intentionally resets optimizer state, record a different training phase rather than calling it equivalent continuation.