A Kubernetes Pod Is Running. Why Is Its Custom Readiness Gate Still False?

A Pod whose containers are up but whose Ready condition remains false is often alarming. The contract for a Pod to be Ready is concrete: all container readiness probes must pass and all custom readiness gate conditions must be true. In this case, we observe Pods in phase Running, containers ready, but a custom readinessGate condition still false. The bounded question is: is that state permissible by design, or does it indicate a breakage?

This playbook treats the symptom “Pod running but custom readiness gate false” as a narrow acceptance-test problem. We will build a pinned-version local cluster and a disposable synthetic fixture with a Pod and Service. We compare an expected-case (Pod Ready passes normal readiness) with the test-case (Pod with a readinessGate) and controls. For each test we list the test case, oracle (expected state), rejection condition, observed evidence (verbatim as produced), and finally mark unexecuted outputs as EXPECTED / NOT EXECUTED. We keep an evidence ledger for each test and a final decision table (ACCEPT / REPAIR / RERUN / HOLD).

Our scope is strictly technical verification of Kubernetes behavior for readinessGates and EndpointSlice admission (Service endpoint inclusion). We follow Kubernetes 1.37 (current stable in late 2026) as the fixture version, using official docs as authoritative contracts. We assume no production secrets or resources; all inputs (manifests) are synthetic. All credentials and external dependencies are excluded. Assumptions: a Linux (e.g. Ubuntu 22.04) host with Docker/Containerd, kind v0.33.0, and kubectl matched to the cluster version. Tests are on a single-control-plane cluster (kind) with one worker node.

1 Define the exact admission contract

We begin by stating precisely what must be true for acceptance. Kubernetes defines Pod readiness via conditions: Pods have many conditions (PodScheduled, Initialized, ContainersReady, etc.) and a Ready condition indicating service eligibility. Notably, Ready is true only if ContainersReady is true and all custom readiness gate conditions are true. If a PodSpec has readinessGates, the kubelet looks for matching PodCondition entries in status; if any is missing or False, the Pod’s Ready stays false. Crucially, “If Kubernetes cannot find such a condition in .status.conditions, the status of the condition is defaulted to False”. Thus our contract:

• Container-level readiness: Each container must succeed its readinessProbe (or have none) to count as ready.

• Custom readiness gates: Each spec.readinessGates[].conditionType must appear in the Pod’s status.conditions with status: True for the Pod Ready to turn True. By default, absent entries count as False.

• Service endpoint (EndpointSlice) admission: A Service’s EndpointSlice will include a Pod’s IP with serving=true only when the Pod’s Ready condition is True. (If spec.publishNotReadyAddresses=true, that affects EndpointSlice conditions, but here we use default false.)

Put another way, the acceptance criteria are twofold: (1) Pod Ready cond = True only when containers are all ready and custom gates all True; (2) the Service’s EndpointSlice must reflect the Pod’s Ready state (serving == True when Ready). Any discrepancy (e.g. Pod Ready False despite healthy containers, or service missing a ready Pod’s IP) indicates a violation.

This follows the style of rigorous acceptance playbooks: we will test each statement independently. As a check, one can compare to Refonte’s preStop hook acceptance playbook: they emphasize defining acceptance exactly (when deletion started, hook finished, etc.) rather than trusting a command success. Likewise, we separate observed state from contract claims. We do not accept “kubectl delete returned no error” as proof of graceful shutdown; similarly, we will not take “Pod is Running” as proof of readiness. The contract is captured by documented Kubernetes behavior and we will verify it with tests.

2 Capture baseline versions and state

Before any tests, we record the environment and Kubernetes version to make results reproducible. We pin exact versions for the control plane and tools. In particular, Kubernetes 1.37.0 is our target (latest stable branch in Nov 2026). We use kind v0.33.0 with its explicit Kubernetes v1.37.0 node image (using the recommended image digest). We capture outputs from kubectl version, uname -a, and tool versions as ground truth.

set -eux
mkdir -p evidence

# Record host and tool versions
date -u +"%Y-%m-%dT%H:%M:%SZ" > evidence/run-timestamp.txt
uname -a > evidence/host-uname.txt
docker version > evidence/docker-version.txt 2>&1
kind version > evidence/kind-version.txt
kubectl version -o yaml > evidence/kubectl-version.yaml

# Create a pinned kind cluster with K8s v1.37.0
KIND_NODE_IMAGE="kindest/node:v1.37.0@sha256:a1ed56cfb0e7b93589bdf97c8cd566405a265939e3620fc4f5de89adff580ae5"
kind create cluster --name readiness-test --image "$KIND_NODE_IMAGE"

kubectl cluster-info > evidence/cluster-info.txt
kubectl get nodes -o wide > evidence/nodes-wide.txt
kubectl get nodes -o json > evidence/nodes.json

EXPECTED: The above commands (not executed here) would produce evidence files. For example, kubectl version should show Server Version: v1.37.0 (or a later patch like v1.37.1, but our image is exactly v1.37.0). kind version output should mention kind v0.33.0, and the kind node image is fixed by digest. We retain these evidence files for each test’s result.

Pin environment and versions (Test case)

Test case: Establish controlled environment with known versions.
Oracle (expected): All captured versions match pins, e.g. kubectl version reports v1.37.x; kind version v0.33.0; Linux kernel etc recorded.
Rejection condition: Any mismatch or missing data (if, say, kind create cluster used latest without digest, or kubectl a different version, we would reject the evidence as not matching the contract).

# Not executed: illustrate expected capture
echo "Cluster created; evidence files generated."

• Outcome: If actual outputs exist, they are archived. Since we cannot execute here, mark outputs as NOT EXECUTED. We will rely on logs like those in the preStop hook acceptance playbook for expected content (for example, that playbook shows using the kind image digest, which we will do similarly).

This setup is our oracle environment. All further tests (Pods, Services) will assume these baseline versions.

3 Build a disposable isolated fixture

We now create a minimal fixture: one Namespace, one Deployment (or Pod) with a readiness probe and a Service to produce EndpointSlices. This fixture is entirely synthetic and will be destroyed after tests. We pin the fixture to this cluster. The point is not the application itself but to isolate readiness behavior.

Test case: Synthetic Pod “example-app” with a readiness gate.
Fixture design: We create a namespace readiness-test, a Deployment (or Pod) example-app with one container (using a lightweight image like k8s.gcr.io/pause:3.9 or a simple HTTP server). We set spec.readinessGates to a custom condition type, e.g. "example.com/custom-ready". We expose it via a headless Service (or normal Service) to trigger an EndpointSlice.

Example manifest (expected-case vs test-case will differ in readinessGates):

# 01-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: readiness-test
---
# 02-pod-with-gate.yaml (test-case Pod)
apiVersion: v1
kind: Pod
metadata:
  name: example-app
  namespace: readiness-test
spec:
  readinessGates:
  - conditionType: "example.com/custom-ready"
  containers:
  - name: app
    image: registry.k8s.io/pause:3.9
    # No readinessProbe means the container is Ready once running
---
# 03-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: example-svc
  namespace: readiness-test
spec:
  selector:
    app: example-app
  ports:
  - name: http
    port: 80

The container pause:3.9 immediately starts and stays Running; without a readinessProbe, Kubernetes considers it ready (it has no conditions blocking it). However, because we set a readinessGate, the Pod’s .status.conditions.Ready will remain False until a matching PodCondition "example.com/custom-ready" turns True. We label this Pod with app: example-app so the Service selects it (creating an EndpointSlice for example-svc).

We deliberately separate the expected-case manifest from this test-case manifest. The expected-case (next section) will have no readinessGate (or a gate that is immediately satisfied). This ensures we don’t unknowingly compare to a manifest that already fails.

For an example of building such a synthetic cluster and labeling nodes, see the Refonte topology playbook, which similarly creates a disposable cluster and captures version info.

4 Create an independent expected-case manifest

To have a valid baseline, we also define an expected-case Pod that is supposed to be ready. For example, a Pod identical to above but without any readinessGate:

# expected-case.yaml
apiVersion: v1
kind: Pod
metadata:
  name: expected-app
  namespace: readiness-test
  labels:
    app: expected-app
spec:
  containers:
  - name: app
    image: registry.k8s.io/pause:3.9
    # No readinessGate, no readinessProbe

In this expected-case, the Pod should transition to Ready as soon as the container is running, because there are no custom gates to hold it. We apply it to verify the normal path:

kubectl apply -f 01-namespace.yaml
kubectl apply -f expected-case.yaml

Oracle: The expected-app Pod should reach Ready=True (ContainersReady=true, no gates). A Service pointing to it (if any) would include it as serving.

Now define the negative control under this same test case:

Preserve a negative control

Test case: A Pod with a readinessGate but no controller to set it, as above in 02-pod-with-gate.yaml.
Oracle: When we apply this Pod, the container starts (ContainersReady=True) but the custom condition is missing, so Ready=False. The EndpointSlice should show serving=false for its endpoint.
Rejection condition: If the Pod unexpectedly becomes Ready without the gate (which would be a bug in kubelet), or if the EndpointSlice erroneously marks the endpoint as serving.

kubectl apply -f 02-pod-with-gate.yaml
kubectl wait --for=condition=ContainersReady pod/example-app -n readiness-test --timeout=60s
kubectl get pod example-app -n readiness-test -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{"\n"}{end}'
# EXPECTED:
# PodScheduled=True
# PodReadyToStartContainers=True
# Initialized=True
# ContainersReady=True
# Ready=False

# Check EndpointSlice for the service:
kubectl get endpointslice -n readiness-test -l kubernetes.io/service-name=example-svc -o yaml
# EXPECTED:
# The listed endpoint (example-app’s IP) should have conditions.serving=false 
# (since Pod Ready is false) and conditions.ready=false.

All commands above are not executed here; we label expected outputs. We expect the service’s EndpointSlice (if the service is applied) to either be empty or include the Pod IP with serving:false (per the EndpointSlice documentation’s mapping of serving to Pod Ready). If this expected control fails (e.g. Pod unexpectedly became Ready), then the contract is broken.

By preserving this negative control, we ensure that the only difference between expected and test cases is the readinessGate. (We could also test a Pod with an irrelevant extra condition type: if spec.readinessGates refers to "foo", that missing condition is default false too.)

5 Reproduce the misleading success

Now we demonstrate the “misleading success” scenario: the Pod is Running and its container appears ready, but it is not actually eligible for service due to the gate. We inspect the Pod status in detail.

# Diagnose the Pod's status conditions and container states
kubectl get pod example-app -n readiness-test -o wide
# EXPECTED: 
# The output will show READY 1/1, STATUS Running, etc., because the container is running and no restarts.
# But see conditions below.

kubectl get pod example-app -n readiness-test -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{"\n"}{end}'
# EXPECTED:
# PodScheduled=True
# PodReadyToStartContainers=True
# Initialized=True
# ContainersReady=True
# Ready=False   # <--- key: all containers ready, but Pod Ready is False due to gate

kubectl get pod example-app -n readiness-test -o jsonpath='{.status.containerStatuses[0].ready}'
# EXPECTED: "true" (container reports ready)

Note how these outputs illustrate the puzzle: ContainersReady=True but Ready=False. By design, this happens because of the custom gate (no condition set). We would not be alarmed if we see ContainersReady=True; Ready=False when a gate is specified. The acceptance condition here is the Pod’s Ready should remain false unless that gate is explicitly satisfied.

For completeness, verify the EndpointSlice again:

kubectl get endpointslice -n readiness-test -l kubernetes.io/service-name=example-svc -o jsonpath='{.items[0].endpoints[*].conditions}'
# EXPECTED:
# The conditions for this endpoint should include "serving":false, "ready":false.

Because the Pod is not Ready, the EndpointSlice marks it serving=false. (If publishNotReadyAddresses were true in the Service, ready might be true even if not serving, but here defaults apply.) If the EndpointSlice showed serving=true, that would violate documented behavior.

6 Run the comparison case

Next, we create a scenario where the custom gate is satisfied, to see the difference. For example, we simulate an external controller by manually patching the Pod status to add the condition.

Test case: Patch the Pod’s status to set example.com/custom-ready=True.
Oracle: The Pod’s Ready should now become True, and the EndpointSlice should mark the endpoint as serving.
Rejection condition: If after patch, Pod Ready remains false (bug: status update not applied or ignored) or if the endpoint is still not serving, then contract fails.

# Simulate the external controller setting the custom condition
kubectl patch pod example-app -n readiness-test --type=json -p='[{"op": "add", "path": "/status/conditions/-", "value":{"type":"example.com/custom-ready","status":"True","lastTransitionTime":"2026-11-13T00:00:00Z"}}]'

kubectl get pod example-app -n readiness-test -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{"\n"}{end}'
# EXPECTED:
# PodScheduled=True
# PodReadyToStartContainers=True
# Initialized=True
# ContainersReady=True
# example.com/custom-ready=True
# Ready=True   # Now Ready flips true because all gates satisfied

kubectl get endpointslice -n readiness-test -l kubernetes.io/service-name=example-svc -o jsonpath='{.items[0].endpoints[*].conditions}'
# EXPECTED:
# Now "serving":true and "ready":true for the endpoint (Pod is now considered serving).

(If the patch command were disallowed by RBAC, an actual test fixture would run an operator instead. For this playbook, we assume a permissive test environment. The outcome should still meet the oracle conditions.)

We now have a comparison case: the baseline Pod (no gate) vs Gate=false vs Gate=true. We see clearly that only the gate difference changed the Ready state, as expected by contract. This test confirms that the kubelet honors the readinessGate logic in both directions.

Detect missing or duplicated evidence

To ensure our evidence is not flawed, we check that no unexpected duplication occurred. For example, patching Pod status should not create duplicate conditions if one already existed. We would inspect .status.conditions before and after to ensure exactly one entry per type.

kubectl get pod example-app -n readiness-test -o yaml | yq '.status.conditions'
# EXPECTED:
# A list of unique condition types.  There should be exactly one entry with "type: example.com/custom-ready".

Additionally, we check EndpointSlice contents: Kubernetes may duplicate an endpoint across multiple EndpointSlices (see docs), but they should not conflict. We can list all slices:

kubectl get endpointslice -n readiness-test -l kubernetes.io/service-name=example-svc
# EXPECTED:
# There may be 1 or more EndpointSlice objects, but collectively they should list the Pod IP exactly once (with serving true in one slice).

A duplication (Pod IP in two slices) is not a correctness issue by itself (EndpointSlice controller might split). What matters is that the Pod’s serving status is correctly reflected at least once.

7 Test negative and boundary controls

We add additional controls to catch edge cases:

• Negative control: Pod with wrong readinessGate type. For instance, a Pod with readinessGates: [{"conditionType":"example.com/other"}] will also end up Ready=False by default. We expect the same behavior (no difference from missing condition).

• Boundary control: Pod with two containers, only one has a readinessProbe. This tests “ContainersReady” vs “Ready” separation. Or test a case where container readiness is delayed.

Example negative control YAML:

# 04-pod-wrong-gate.yaml
apiVersion: v1
kind: Pod
metadata:
  name: bad-gate-app
  namespace: readiness-test
spec:
  readinessGates:
  - conditionType: "example.com/typo-gate"
  containers:
  - name: app
    image: registry.k8s.io/pause:3.9

Apply and test:

kubectl apply -f 04-pod-wrong-gate.yaml
kubectl wait --for=condition=ContainersReady pod/bad-gate-app -n readiness-test --timeout=60s
kubectl get pod bad-gate-app -n readiness-test -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{"\n"}{end}'
# EXPECTED:
# Similar pattern: ContainersReady=True, Ready=False (because example.com/typo-gate is absent and defaults to False).

This confirms that any missing custom gate yields the same false result. If somehow the wrong gate were ignored and Pod became Ready, that would contradict the documented readiness-gate behavior.

For a boundary control, imagine if the Pod had two containers and one is not ready (e.g. a failing init or missing readinessProbe). Then even if the gate is satisfied, ContainersReady=False would keep Ready=False (the gate cannot override a failing container). That is part of contract but not our main focus. However, we ensure we do not mix up those conditions.

8 Collect per-case raw evidence

For each test scenario, we collect raw outputs into an evidence ledger for traceability. For example, for each Pod (expected-app, example-app, bad-gate-app) and each step, we might record:

We do not have actual timestamps, so we mark expected. The key is that each line is evidence of the state after each manipulation. For instance, at T1, the evidence should show the Pod has example.com/custom-ready=False (or absent) and Ready=False, and the EndpointSlice reflects that. At T2, the evidence would show example.com/custom-ready=True and Ready=True, and the EndpointSlice shows serving=true. This aligns with the acceptance claims (admission = gate True, adoption = endpoint becomes active).

All entries above are EXPECTED / NOT EXECUTED in text: actual command outputs would fill the placeholders. This ledger is our acceptance artifact, not the raw terminal transcript. It is built from the outputs we would obtain.

Check the actual state after failure

If any expected state failed (e.g. Pod Ready unexpectedly True), we would inspect deeper. For example, if example-app had somehow become Ready despite no patch, we’d look at event logs, kubelet logs, or maybe an intervening controller (but none exists here). We might run: kubectl describe pod example-app to see if anything set the condition (unlikely). We’d verify no finalizers or old status from a recycled name.

Since our tests are not actually run, we mark these checks as EXPECTED / NOT EXECUTED as well. In a real run, if a failure occurred, this is where we would gather container logs, kubelet logs, etc., to reconcile the discrepancy.

9 Reconcile identities and counts

Finally, we correlate Pod identities and count of endpoints. We ensure that the number of IPs in EndpointSlices matches the number of Pods we expect, and that each Pod’s UID and name are accounted for.

For instance, if we create 3 Pods (expected-app, example-app, bad-gate-app) all selected by the same Service (using app: example-app and app: expected-app labels appropriately), we can verify:

kubectl get pods -n readiness-test -o name
# EXPECTED: list of pod names: example-app, expected-app, bad-gate-app

kubectl get endpointslice -n readiness-test -l kubernetes.io/service-name=example-svc -o json
# EXPECTED: all endpoints corresponding to "app=example-app" (if we had 2 pods labeled app=example-app) 
# or service name example-svc slices should list the matching Pod IP(s).

For simplicity in this playbook, we treated each Pod separately (the Service only selected example-app). If we wanted to test duplicates (e.g. 3 replicas), we’d check counts. The EndpointSlice concept says endpoints may appear in multiple slices; clients must dedupe. Our evidence ledger for the relevant test does not need to reflect duplications beyond confirming presence or absence.

At this point we have verified the identity mapping: each Pod (example-app, etc.) yields an EndpointSlice entry with serving equal to its Ready status. The counts and UIDs (if we captured them) match. We would record in the ledger something like “Pod UID x appears with serving=false” and “same Pod appears in slice owning Service=example-svc”.

See the Refonte ConfigMap adoption playbook for an example of reconciling object identity over time (it uses a ledger of Pod UIDs and observed values across updates). Here we similarly would join Pod UID, readiness condition, and endpoint presence.

10 Explain documented behavior and limits

At this stage we consolidate what Kubernetes documentation tells us about this behavior and any caveats.

By design, a custom readinessGate must be managed by something external. The API docs state: “Your application can inject extra feedback or signals into PodStatus: Pod readiness. To use this, set readinessGates in the Pod’s spec… This field does not automatically update. The kubelet only uses those gates as additional criteria.” In plain terms: Kubernetes will not auto-set a custom gate, and absence=“False”. Thus it is expected (and documented) that our test-case Pod stays unready unless the gate is updated. The only way to satisfy such a gate is for an operator/controller or manual command to add a matching PodCondition.

On the EndpointSlice side, docs say that for Pod endpoints, conditions.serving mirrors Pod Ready. A Pod that is Ready turns on serving=true; if not, serving=false. We have seen this consistent mapping in tests. The ready condition in EndpointSlice is simply serving && !terminating. If the Service spec had publishNotReadyAddresses, the behavior would differ (the ready condition would be true even if serving was false) but we used the default (publishNotReadyAddresses=false) for simplicity.

Limitations: This acceptance test focuses on the control-plane logic of readinessGates. We assume no flakiness or race conditions (e.g. our manual patch is instantaneous). In a real cluster, updating status.conditions often requires using a client library or Kubernetes API directly (as kubectl patch --type=json to status might fail without proper field-manager). We also assume no conflicting controllers are modifying the same Pod. If there were a more complex system (e.g. an Ingress controller injecting readiness gates), additional factors come into play, but our contract remains the same: the Pod cannot be considered Ready by the kubelet unless its custom conditions are True.

If any behavior deviates (for example, if the Pod unexpectedly enters Ready without our intervention, or endpoints leak in), the deviation would be a bug or misconfiguration. According to the Pod lifecycle and EndpointSlice documentation, such deviations are violations of documented behavior.

Bound the repair to the contract

Any repair we propose must satisfy the original contract (all containers ready & all gates True) and not do anything extraneous. For example, deleting the Pod might make it “go away” but that does not honor the contract for the running Pod. Similarly, if the gate condition was wrong (typo), renaming the Pod is not a reasonable repair. The repair must either ensure the gate becomes true, or remove the gate if it was not intended (but removing a gate is a contract change, so only acceptable if the team agrees).

Hence, we will repair just the readiness condition. We must avoid altering container configuration or introducing side effects.

11 Apply the smallest safe repair

Following the principle of least change, the minimal fix is to set the custom condition to True. In a live system that means running the external controller or operator that satisfies the gate. For our test, we simulate this by a patch (as we already did above). We now consider that “the condition is supposed to be True.” The evidence after that patch (T2) showed the Pod became Ready and endpoint serving=true.

We document the repair action (patch to status) as our chosen fix. In a production fix, one might instead deploy or fix the controller that sets the condition. Importantly, this repair does not change any contract fields (pod spec) or restart containers. It simply finalizes the Pod’s state per the existing contract.

# Repair action (already simulated above):
# kubectl patch pod example-app ... to set custom-ready=True
# Now expect:
kubectl get pod example-app -n readiness-test -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'
# EXPECTED: "True"
kubectl get endpointslice -n readiness-test -l kubernetes.io/service-name=example-svc -o jsonpath='{.items[0].endpoints[*].conditions.serving}'
# EXPECTED: "true"

We then verify the repair: the Pod’s Ready flips to True, and the service endpoints reflect that. We include these outputs in the evidence ledger (already under T2). Since our patch is the fix, we have now achieved contract-compliance.

12 Test rollback and rerun safety

We now ensure our fix is robust. If we “undo” the fix or rerun it, behavior should be predictable.

Rollback test: Simulate the condition becoming invalid again (e.g. revoke permission, or if we had removed a Pod label). In our context, we revert the status change:

# Rollback: remove the custom condition we added
kubectl patch pod example-app -n readiness-test --type=json -p='[{"op": "remove", "path": "/status/conditions/5"}]'
kubectl get pod example-app -n readiness-test -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{"\n"}{end}'
# EXPECTED:
# Now custom-ready is absent again, so Ready=False. (ContainersReady still True)

This should return the Pod to the original failing state (T1). The EndpointSlice would again show serving=false. If rollback does not revert (e.g. status subresource restrictions), at least a pod restart should. But our intended rollback reveals that the system can move backward as expected: removing the gate True causes Ready to False again. This validates that there is no one-way latch.

Rerun test: Reapply the patch (setting True) after rollback. We should get back to Ready=True (T2 state again). Doing this repeatedly should be idempotent.

# Rerun repair:
kubectl patch pod example-app -n readiness-test --type=json -p='[{"op": "add", "path": "/status/conditions/-", "value":{"type":"example.com/custom-ready","status":"True","lastTransitionTime":"2026-11-13T00:01:00Z"}}]'
kubectl get pod example-app -n readiness-test -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'
# EXPECTED: "True"

We confirm that toggling the condition behaves as a toggle. We also ensure no unexpected side effects: container ID, Pod UID, Service objects remain unchanged, only status bit changed.

Assign operational ownership

Who should own this issue? The readiness gate was requested by someone (application or platform team). If the application defined the gate, then the application team (or their operator/controller) must satisfy it. If the platform introduced the gate (e.g. a mesh sidecar readiness), then platform ops must handle it. In either case, our evidence shows it is not a container crash or node issue, but a missing condition update. The repair (a status patch) would require elevated privileges (or a controller). Thus the likely owner is a controller process tied to the owner of the gate type (e.g. an AWS/ALB controller, an Istio injector, or custom operator). We do not proceed to restart the Pod as a fix (that would not guarantee the condition is set). Instead, ops should fix the controller logic.

Our acceptance test has pinned the minimal fix action. The “owner” owes us either a working controller or removal of the gate if it was in error.

Refonte’s ConfigMap update playbook similarly distinguishes platform vs app responsibility when a contract is not met (i.e. whether stale config is app cache vs platform failure). Here, the “stale” state is the Pod not serving traffic; the owner who defined the readinessGate must satisfy it.

13 Decide ACCEPT, REPAIR, RERUN or HOLD

We summarize decisions for the observed failure mode “Pod running, readinessGate false” in a decision matrix:

• ACCEPT: We accept the situation if it matches documented design. E.g. if containers are not ready or the gate is false, it’s correctly not serving; no acceptance needed until gate is satisfied. If somehow everything is ready and yet the Pod was not serving without a gate (which shouldn’t happen here), maybe accept but log.

• REPAIR: We repair when the state violates an implicit expectation. In this scenario, the contract expects that if the Pod has been granted permission to receive traffic (through the readiness gate), it should be ready. If the gate type was incorrectly spelled or the controller missing, that is a misconfiguration or bug, so repair by fixing the gate logic. If multiple container or readinessProbe issues, fix those. The minimal safe repair in our case was exactly the status patch (or controller fix) that sets the condition to True.

• RERUN: We would re-run tests after repair to confirm. If the Pod is still failing readiness after fix, we might teardown and recreate it. Idempotency requires re-running (we showed that patching twice yields same outcome).

• HOLD: We hold if the issue is out of our scope or awaiting external action. For instance, if the readiness gate is purposefully waiting for a manual signal (say a human triggers a flag), we note that the situation is expected until that event. Or if the controller is temporarily down, we wait. But even then, a Timeout policy might apply. In our test, there was no legitimate hold condition: either we fix or document the expected wait.

In this case, our outcome is REPAIR: we identified the correct fix (set gate True), applied it (in test), and verified. The Pod has rejoined the service. If this were production, we would implement the controller fix and rerun (RERUN) to confirm.

We do not accept the failing state as final, because the contract called for readiness once the external condition is met. And we do not hold indefinitely: we have a patch deadline for integration. Thus the final disposition is to fix and then validate.

14 Develop related skills with Refonte Learning

This incident highlights the importance of understanding Kubernetes contracts and testing them. Engineers can hone these skills in structured programs. For example, Refonte Learning’s DevOps Engineering Program is designed to teach container orchestration rigorously. It is a 3-month course (12–14 hours/week) for students or professionals with at least a bachelor’s enrollment. The curriculum explicitly covers Docker and Kubernetes (as well as CI/CD, cloud, Terraform, etc.), matching the technologies in play here. The program includes real-world projects, virtual internships, and mentorship (led by seasoned engineers like Oskar Eriksson). Graduates earn training and internship certificates. Those seeking to deepen their practical Kubernetes troubleshooting and validation skills would find it valuable training.