A self-hosted runner fleet can look healthy right up to the moment it fails in two different ways. Existing workers may still show online and finish jobs, while a newly scaled instance built from an older image cannot register. A disaster-recovery template can reintroduce a stale binary after the main pool has been upgraded. A critical security release can cause job queuing to pause before a monthly maintenance window arrives. None of those conditions is visible from a simple runner-count dashboard.
GitHub changed the operating surface on September 3, 2026 by publishing a version-specific deprecation REST API. The GitHub Actions early September 2026 update says the service can return the registration and runtime end dates for a specified runner version at repository, organization, or enterprise scope. The distinction matters because registration failure and runtime failure are separate operational events with different blast radii and recovery paths.
A production GitHub Actions runner lifecycle therefore needs more than patching. It needs version discovery, image provenance, deprecation lookups, a moving 30-day update deadline, critical-security triggers, ARC governance, upgrade rings, spare capacity, and rollback to a version GitHub still accepts. Treat the runner binary as a time-bounded dependency embedded in images, containers, templates, scripts, caches, and recovery artifacts. The goal is to prevent unsupported runners from silently returning after one successful upgrade.
What the Runner Deprecation API Changes for Fleet Operations
The new API turns runner lifecycle from a release-note interpretation problem into a machine-readable control. According to GitHub Docs for self-hosted runner REST endpoints, the end-of-life lookup is version-specific. It does not enumerate your fleet and it does not return every version in one call. Operators must first discover distinct versions in use, then query the appropriate repository, organization, or enterprise endpoint for each version.
That architecture is important. A version that appears on one organization pool may also be baked into an enterprise image catalog, a repository-specific specialist runner, or an ARC scale set. The deprecation response belongs beside, not instead of, the inventory. Treat runner_version, registration_deprecates_at, and runtime_deprecates_at as lifecycle attributes to join to a runner record, image digest, template revision, owner, and capacity plan.
Control question | GitHub data | Fleet evidence | Operator decision |
What is running? | Runner list APIs expose version, status, busy state, OS, labels, ephemeral flag | Runtime telemetry and controller state | Discover distinct versions and active capacity |
What will expire? | Version deprecation API | Stored response with query time and scope | Set rebuild and replacement deadlines |
What can relaunch? | Not answered by runner API alone | VM images, container tags, launch templates, ARC values, caches | Remove stale sources before cutover |
Who is affected? | Scope and runner group/repository assignments | Workflow routing and ownership map | Choose ring, stop condition, and escalation path |
This article stays on fleet lifecycle rather than repeating a broad GitHub Actions versus Jenkins platform comparison. The operational question here is narrower: given that GitHub Actions is already in use, how do you keep every self-hosted execution path on a supportable runner version without sacrificing delivery capacity or rollback safety?
· Vendor-documented behavior: deprecation is queried for a specified version at repository, organization, or enterprise scope.
· Practitioner control: run discovery before deprecation lookup, because the lookup does not discover hidden versions in templates or caches.
· Evidence gate: retain the queried scope, version, timestamps, response fields, image provenance, and remediation owner together.
Define Registration Deprecation and Runtime Deprecation Separately
Registration deprecation answers whether a runner version can configure or reconnect to the service. Runtime deprecation answers whether an already connected runner can continue receiving and executing jobs. GitHub's June 12, 2026 minimum-version enforcement timeline makes the split concrete: the registration floor and the moving job-execution floor are not the same control.
A persistent runner can therefore be in a deceptive state. It may have registered weeks earlier and still appear online, while an identical instance launched from the same image can no longer register. Later, the older connected worker can also stop receiving jobs when it falls below the runtime floor. A restart or reconfiguration can turn a runtime-only risk into an immediate registration failure. Operators should model those transitions explicitly instead of assigning one generic end-of-life date.
State transition | What changes | Likely symptom | Required evidence |
New registration | Fresh host or pod runs config | Registration succeeds or is rejected | Runner version, registration response, image digest |
Re-registration | Existing host loses or recreates registration | Previously working node cannot reconnect | Prior version plus reconfiguration event |
Persistent connection | Runner remains registered | Dashboard may remain green | Heartbeat plus version and deprecation headroom |
Restart | Service/process reconnects | Failure appears after maintenance | Restart timestamp and registration logs |
Job pickup | Service dispatches queued work | Jobs queue despite online hosts | Queue age, runtime version, job annotations |
Critical security pause | GitHub pauses queuing until update | Sudden loss of eligible workers | Security release signal and version state |
Map Every Failure State to an Operator Action
Fleet state | Meaning | Immediate action | Exit condition |
Supported | Registration and runtime dates have acceptable headroom | Continue scheduled discovery | Next check completes with no new risk |
Approaching registration cutoff | New or restarted instances are at risk soon | Freeze stale image promotion; build replacement | Replacement registers in target scope |
Registration blocked | Version cannot newly register or re-register | Stop autoscaler churn; redirect to supported pool | Supported capacity replaces blocked launch path |
Runtime cutoff approaching | Existing workers may soon stop jobs | Drain by ring and reserve capacity | Supported fleet can absorb target concurrency |
Runtime blocked | Existing runner no longer executes work | Quarantine, replace, and reduce concurrency if needed | Queue drains on supported workers |
Critical-update paused | Job queuing paused until update | Emergency rebuild/update path | Patched version executes validation job |
Offline | Runner unavailable for another reason | Diagnose separately from deprecation | Healthy heartbeat or intentional retirement |
Unknown | Version or schedule unavailable | Treat as risk, fix telemetry/permissions | Version and deprecation evidence become known |
The key operational rule is simple: green status is not a proof of future eligibility. Registration tests prove that replacement capacity can join. Job-execution tests prove that eligible workers can actually receive work. Both are required acceptance gates.
Build a Complete Runner Inventory Across Repository, Organization, and Enterprise
Start with the runner APIs because they expose live state. Current organization and repository list responses include runner ID, name, operating system, status, busy state, ephemeral flag, version, and labels. Then expand the record with runner groups and repository assignments. The GitHub REST endpoints for self-hosted runner groups provide the group relationships needed to understand which pools are security or routing boundaries.
Do not stop there. GitHub's enforcement guidance says registration audit events include runner version, but it also warns that audit logs are not a complete inventory of connected runners. Audit events tell you what registered, not every artifact that can create a runner tomorrow. Reconcile runtime state with image factories, container registries, autoscaling templates, ARC custom resources and Helm values, bootstrap scripts, disaster-recovery automation, manually maintained hosts, and caches on nodes that can launch old containers.
Field | Example meaning | Source of truth |
runner_id / name | Concrete GitHub runner identity | Runner list API |
scope | Repository, organization, or enterprise owner | Inventory job configuration |
group / labels | Routing and access boundary | Runner group API plus labels |
OS / architecture | Compatibility class | API plus image metadata |
version | Runner binary version | API or local runtime telemetry |
ephemeral | One-job or persistent lifecycle | API / controller configuration |
image digest / template revision | Reproducible source artifact | Image registry or VM catalog |
last job / busy / queue | Operational use and pressure | Actions and monitoring telemetry |
registration/runtime dates | Support deadlines | Deprecation API |
owner / capacity / next action | Accountability and risk treatment | Internal fleet registry |
Ownership should follow the boundary between platform and application responsibility. A central platform team owns runner software, images, ARC controllers, network and secret boundaries, and shared capacity. Application teams own workflow compatibility, required labels, business validation, and maintenance windows. Security owns critical-update escalation and image policy. SRE or operations owns incident command and queue recovery. Executive risk acceptance is reserved for time-bounded exceptions where the supported path cannot be restored before a known cutoff. The broader platform engineer versus DevOps engineer discussion can help define those organizational boundaries, but the runner registry must make them explicit per pool.
· Reject inventory rows with an unknown version, unknown owner, or unknown installation source.
· Tag every image or template with an immutable identifier, not only a mutable alias such as latest.
· Link each runner group to the repositories and workload classes that would queue if that group lost capacity.
· Record manual or break-glass launch procedures because they are frequent sources of stale binaries during incidents.
Query Version and End-of-Life Data Programmatically
The safe automation pattern is inventory first, distinct versions second, deprecation lookup third. Use the narrowest scope that matches the runner's registration boundary. Repository-level runners should use the repository path, organization pools the organization path, and enterprise pools the enterprise path. GitHub documents separate permission requirements, so a central job may need more than one credential or a broker that can query each scope without granting write access.
# Example: discover versions at organization scope
gh api --paginate '/orgs/$ORG/actions/runners?per_page=100' \
--jq '.runners[] | [.id,.name,.version,.status,.busy,.ephemeral] | @tsv'
# Query the schedule for one discovered version
gh api '/orgs/$ORG/actions/runners/deprecations/$VERSION' \
-H 'X-GitHub-Api-Version: 2026-03-10'
# Equivalent path shapes for other scopes
/repos/$OWNER/$REPO/actions/runners/deprecations/$VERSION
/enterprises/$ENTERPRISE/actions/runners/deprecations/$VERSION
Do not infer a missing field. Current documentation text says the response includes both registration and runtime dates, while some rendered examples show only runner_version and runtime_deprecates_at. Treat a missing registration_deprecates_at or runtime_deprecates_at as unknown evidence, not as unlimited support. Store the raw response, status code, query timestamp, API version header, and scope so the discrepancy can be reviewed without rerunning the query.
1. Enumerate live runners and distinct versions for each managed scope.
2. Query each distinct version at the same scope that governs registration.
3. Join results to image digests, templates, ARC releases, owners, groups, and capacity.
4. Flag missing fields, permission failures, or HTTP errors as inventory exceptions.
5. Persist a dated snapshot so changes in future API responses can be audited.
Normalize API Results Into a Fleet Risk Register
Risk register field | Derived value |
registration_headroom | registration_deprecates_at minus evaluation time |
runtime_headroom | runtime_deprecates_at minus evaluation time |
internal_deadline | Earlier of policy deadline, registration risk date, runtime risk date, or security trigger |
relaunch_exposure | Count of images/templates/caches that can still emit the version |
capacity_exposure | Eligible concurrency lost if the version becomes unusable |
rollback_eligible | Yes only if rollback version still passes registration and runtime checks |
next_action | Rebuild, drain, replace, investigate, retire, or exception review |
A risk register is more useful than a version spreadsheet because it connects service policy to physical and logical launch paths. The same version can have low risk in an idle lab pool and critical risk in the only group that can reach a production deployment network. Prioritize by business effect and replacement difficulty, not version number alone.
Account for the Moving 30-Day Update Requirement
GitHub's self-hosted runners reference states that self-hosted runners auto-update by default and that disabling automatic updates transfers update responsibility to the operator. The June 12 enforcement notice adds the operational rule: any runner software release, including major, minor, or patch releases, can start the 30-day update window. A runner that is not updated within 30 days can stop receiving queued jobs. A critical security update can cause job queuing to pause sooner until the update is applied.
That means a monthly patch meeting is not a sufficient control. If a runner release appears the day after the meeting, the next meeting may occur at or beyond the service deadline. Use release publication as an event source. The actions/runner release feed showed v2.337.0 as the latest release on August 26, 2026 during the September 8 verification, and it notes that releases are progressive, so the version available to a particular enterprise, organization, or repository can differ. Your automation must confirm the expected downloadable version for the scope rather than blindly baking the highest public tag.
Clock | Vendor rule | Operator policy |
Release publication | Starts the update window | Open lifecycle change automatically |
Internal rebuild target | Not prescribed by GitHub | Set earlier than service cutoff based on validation lead time |
30-day service window | Jobs can stop queuing if update is not applied | Never use day 30 as planned completion |
Critical security release | Job queuing can pause sooner | Escalate to emergency update path |
Progressive release availability | Latest public release may not be offered everywhere immediately | Record expected version from target scope before promotion |
Use an internal maximum lag rather than a universal threshold copied from another company. For example, an operator may choose a policy that leaves enough time for image build, vulnerability scanning, smoke tests, a low-risk ring, and one full rollback cycle before GitHub's cutoff. The exact buffer should be justified by your fleet's lead time and recovery characteristics.
Decide Where Auto-Update Is Safe and Where Images Must Be Rebuilt
Auto-update and image-based update are different reliability models. Auto-update minimizes version lag on mutable persistent hosts, but it changes software in place and can make fleet state diverge from the image factory. Image-based updates preserve determinism for ephemeral containers and virtual machines, but only if the image pipeline can publish and rotate quickly enough. Regulated environments may prefer explicit image promotion because it creates stronger provenance and approval evidence, while small non-privileged pools may accept auto-update if drift is continuously measured.
Runner model | Security / determinism | Capacity risk | Rollback model | Preferred lifecycle |
Mutable persistent host | Lower determinism if binary changes in place | Low if update succeeds, higher if many restart together | Host snapshot or package rollback, plus eligibility check | Auto-update or managed in-place ring |
Immutable VM image | High provenance if digest/revision is fixed | Depends on replacement and warm-up rate | Repoint template to supported prior image | Rebuild and rotate |
Ephemeral container | High if image digest is pinned | Image pull and cluster capacity matter | Roll back digest only if still supported | Bake runner into image |
ARC scale set | Controller plus runner image both matter | Listener/controller and pod churn can amplify failure | Helm/config revision plus image digest | Coordinated ARC and image rollout |
Specialized regulated pool | Strong change evidence usually required | Often low spare capacity | Pre-approved supported fallback image | Explicit promotion with exception control |
The decision rule is not 'auto-update good' or 'auto-update bad.' Use auto-update where in-place change is acceptable and telemetry can verify the effective version. Use immutable rebuilds where the execution environment itself is a controlled artifact. In either model, the fleet registry must reconcile actual runner version to intended version after every release.
Govern Runners With --disableupdate
GitHub documents --disableupdate as the registration flag for turning off automatic runner software updates. That flag is appropriate for ephemeral containers because repeated self-update on every new container defeats immutable image design. It is also appropriate where change control requires the binary to enter through an image factory. The tradeoff is explicit: the platform team now owns the update clock.
· Create an automated build trigger when a runner release becomes available to the target scope.
· Set a maximum image lag and alert before that lag threatens the 30-day service window.
· Block promotion of an image whose embedded runner version has insufficient registration or runtime headroom.
· Prohibit mutable tags from being the only reference in production templates; store image digest or immutable image ID.
· Expire exceptions. A waiver without an expiry date converts disabled auto-update into permanent drift.
./config.sh --url <scope-url> --token <registration-token> --disableupdate
The command is only the switch. Governance is the release watcher, build pipeline, promotion gate, owner, deadline, and proof that every launch source has rotated.
Rebuild VM Images, Containers, Templates, and ARC Deployments
A runner upgrade is incomplete until every source that can launch capacity is updated. Start at the source pipeline. Fetch the runner package from the expected scope or verified GitHub release, validate the published checksum, install required dependencies, apply your hardening baseline, scan the result, run registration and job smoke tests, then publish an immutable artifact. The public runner release notes publish SHA-256 checksums for packages, which gives the image factory a direct integrity check before baking.
Image scanning should be part of the runner build, not a substitute for lifecycle evidence. Refonte Learning's overview of Trivy and modern DevOps security tooling is useful background for the scanning layer. The fleet control still needs to prove which runner version the image contains, which dependencies changed, and whether the image can register and execute jobs in the intended GitHub scope.
Build stage | Required control | Stop condition |
Fetch | Pin version and architecture; verify checksum | Checksum mismatch or unexpected package source |
Bake | Install runner plus approved dependencies and certificates | Unapproved dependency drift |
Scan | OS/package/image security scan and policy gate | Blocking vulnerability or policy violation |
Smoke | Start runner in isolated scope, register, execute test job | Registration, label, network, or artifact failure |
Publish | Immutable digest/image ID plus SBOM or build metadata | Missing provenance or mutable-only reference |
Rotate | Update launch template, scale set, ARC values, DR template | Any active launch path still points to stale artifact |
Retire | Remove old aliases and caches after rollback window | Rollback evidence not yet retained |
ARC requires two lifecycle views: the controller deployment and the runner image used by the scale set. GitHub's Actions Runner Controller documentation describes ARC as the Kubernetes operator that orchestrates and scales self-hosted runners. Updating the controller alone does not replace a stale custom runner image. Updating the image alone does not prove the controller, listener, authentication, and scaling behavior remain compatible. Promote both through the same test rings when either changes materially.
· Audit cloud launch templates and autoscaling groups after rotation.
· Audit container registry tags and node image caches, including disconnected or disaster-recovery clusters.
· Audit ARC Helm values, custom runner pod templates, controller versions, and secrets references.
· Audit golden-image galleries, manual runbooks, Packer/Terraform variables, and bootstrap scripts.
· Delete or quarantine stale artifacts only after a supported rollback artifact has been proven.
Design Ephemeral and Persistent Runner Upgrade Paths
Ephemeral and persistent runners fail differently during change. An ephemeral runner should be replaced by changing the source image and allowing new capacity to launch. A persistent runner must be drained, updated or replaced, restarted, revalidated, and returned to service. Treat those paths separately in automation because one is a supply-chain rotation and the other is a host lifecycle operation.
Operation | Ephemeral pool | Persistent pool |
Drain | Stop new pod/VM creation from old template; let active jobs finish | Remove runner from routing group or otherwise prevent new assignments |
Replace | Launch new instances from supported image | Update in place or provision replacement host |
Register | Every fresh instance exercises registration path | Often only exercised after restart or reconfiguration |
Job test | Synthetic workflow per image/revision | Synthetic workflow after service restart |
Retire | Scale stale replicas to zero and purge old source | Deregister and decommission host after evidence capture |
Rollback | Repoint to known-good supported image | Restore supported host/image and registration state |
For persistent hosts, never update every runner in a group at once. Preserve enough supported capacity to absorb normal queue pressure plus the workers being changed. For ephemeral pools, consider image pull latency, cluster scheduling, quota, cloud instance availability, and ARC reconciliation time. A mathematically sufficient max pool size can still fail operationally if the new image cannot pull or the autoscaler is repeatedly launching a registration-blocked version.
Prevent Old Cached Images From Returning
Cached artifacts are the classic reintroduction path. A clean production rollout can be undone by a scale-out event from a stale VM template, a Kubernetes node using an old image already present locally, or a disaster-recovery script that references a superseded version. Closing the incident requires proving that those sources can no longer emit the deprecated runner.
· Replace mutable image aliases in production configuration with immutable digests or image versions.
· Invalidate or age out node caches where policy permits; force a pull when rotating a critical runner image.
· Update standby-region templates and recovery documentation in the same change, not in a later cleanup ticket.
· Search infrastructure-as-code repositories and secret/bootstrap stores for the old runner version string or image identifier.
· After cutover, deliberately scale from zero in a test environment and verify the launched runner reports the intended version.
Acceptance is not 'all currently connected runners are new.' Acceptance is 'every approved launch path produces a supported runner, and no retired path can silently reappear.'
Test Registration, Job Execution, Labels, Groups, and Network Access
A runner image is not production-ready because the process starts. It must prove the full control plane and data path: registration, correct scope, runner group placement, label routing, repository access, reusable workflow behavior, artifact and cache access, secrets boundaries, outbound network, proxy handling, and workload-specific dependencies. GitHub's guide to using self-hosted runners in workflows explains that workflows can target self-hosted runners through labels or groups, so a label mismatch can create a queue even when supported capacity exists.
Test | What it proves | Failure means |
Fresh registration | Version is accepted at target scope and authentication path works | Image or token/permission/network issue |
Re-registration | Restart/recovery path still works | Persistent fleet may fail during maintenance |
Label route | runs-on selection reaches intended runner class | Jobs may queue or land on wrong trust boundary |
Runner group route | Repository/group access policy is preserved | Security boundary or availability regression |
Artifact/cache | GitHub endpoints and credentials function | Build performance or correctness regression |
Secrets boundary | Only expected secrets are exposed | Security regression |
Proxy/network | Required GitHub and internal endpoints are reachable | Runner may appear online but jobs fail mid-step |
Representative workload | Toolchain, service containers, deployment path work | Image is not workload-compatible |
Use a synthetic workflow plus a representative workload from each risk class. The synthetic workflow should report runner name, version, OS/architecture, labels, image revision, and selected network checks as non-secret diagnostics. The representative workload catches toolchain or permission assumptions that a synthetic job misses.
· Stop condition: any registration failure, wrong group/label placement, unexpected secret exposure, or inability to fetch artifacts/caches.
· Rollback condition: supported replacement fails a production-equivalent test and the prior known-good image still has sufficient support headroom.
· Recovery evidence: fresh runner registers, receives the intended job, accesses required services, and reports the expected immutable image identifier.
Create Alerting for Deprecation Windows and Critical Security Pauses
Runner monitoring must correlate lifecycle risk with delivery impact. A version alert without queue data can be noisy. Queue delay without version data can send responders toward capacity when the true cause is deprecation. Combine deprecation dates, actual runner versions, offline/busy state, queued-job age, image age, release publication events, job annotations, and stale launch-source counts into one operational view.
This is where general observability in DevOps principles become concrete. Metrics describe fleet state, logs explain registration and controller errors, traces or workflow timing show where jobs wait, and events capture releases or security notices. The runner lifecycle view should make one question easy to answer: if GitHub rejects this version now, which repositories lose how much eligible capacity and what supported artifact can replace it?
Signal | Alert intent | Example operator response |
Registration headroom | Warn before new/restarted capacity is threatened | Accelerate image rotation and freeze stale source |
Runtime headroom | Warn before existing workers stop taking jobs | Drain by ring and reserve supported capacity |
Release age / update lag | Detect approach to 30-day rule | Trigger rebuild or in-place update |
Critical update event | Bypass normal cadence | Emergency promotion path and incident staffing |
Unknown version/schedule | Detect telemetry or permission blind spot | Repair discovery before accepting fleet health |
Stale image count | Detect reintroduction risk | Rotate templates/caches/DR sources |
Queued-job age | Measure delivery impact | Shift routing, reduce concurrency, restore capacity |
Supported idle capacity | Measure recovery margin | Pause rollout if reserve drops below policy |
Set SLOs and Escalation Thresholds
Service level objectives (SLOs) should be local policy, not invented universal numbers. Define them from your rebuild lead time, workload criticality, pool elasticity, and incident recovery objectives. The important part is to make each threshold actionable and owner-bound.
Metric | Policy question |
Minimum registration headroom | How much time is required to rebuild, test, and replace capacity before new nodes may be blocked? |
Minimum runtime headroom | How much time is required to drain high-risk pools before existing nodes may stop jobs? |
Update latency | How long after an offered runner release may each ring remain on the old version? |
Unknown-version tolerance | How many unknown records are acceptable before fleet health is considered unverifiable? |
Stale-image count | How many launchable artifacts may contain a superseded version? Prefer zero after cutover. |
Queue delay | At what age does delivery impact require traffic shaping or incident command? |
Available capacity | What supported reserve must remain while a ring is drained or rebuilt? |
Owner acknowledgement | How quickly must a responsible owner accept and act on a lifecycle alert? |
Escalate a critical security pause as an incident even if queue depth is initially low. The absence of a backlog can simply mean demand has not arrived yet. Similarly, an unknown deprecation response for a privileged deployment pool is not a monitoring warning; it is a control failure because the platform cannot prove future eligibility.
Roll Out Upgrades by Runner Group and Workload Risk
Runner groups are useful upgrade rings because they already reflect routing or trust boundaries. Use a lab pool first, then low-risk repositories, non-production workloads, general build capacity, privileged deployment runners, specialized hardware, and finally business-critical groups. Do not force that exact sequence if your topology differs; the principle is to increase blast radius only after the previous ring proves registration, execution, routing, and capacity.
Ring | Primary validation | Soak / stop gate | Rollback trigger |
Lab | Registration, update mechanism, diagnostics | Stop on any control-plane failure | Cannot register or execute synthetic job |
Low-risk repos | Real workflow compatibility | Stop on unexpected queue growth or toolchain errors | Failure rate attributable to image/version |
Non-production | Secrets, environments, reusable workflows | Stop on permission or network regression | Deployment path fails |
General workloads | Capacity and autoscaling behavior | Stop if supported reserve erodes | Queue delay breaches local threshold |
Privileged deployment | Tight network/secrets boundary | Stop on any boundary drift | Any access-control regression |
Specialized / critical | Hardware, business validation, recovery | Proceed only with tested fallback capacity | Business validation or recovery test fails |
Worked example: payments deployment runner fleet. An organization discovers three versions across 48 connected runners. The live API shows most general-build workers on a supported current version, six persistent deployment runners on an older version, and eight ephemeral runners created by an ARC scale set from a stale custom container. The image registry also contains a disaster-recovery tag pointing to the same stale digest. The deprecation lookup shows registration risk arriving earlier than runtime risk for that version.
The platform team first rebuilds the custom image with the supported runner binary, verifies its checksum, scans it, then launches a lab ARC scale set. Registration succeeds, but the first deployment test fails because the rebuilt image omitted an internal certificate bundle. That is a stop condition. The team does not roll the image into production and does not fall back to the stale digest automatically. It fixes the image, reruns registration, label, group, artifact, proxy, and deployment tests, then promotes to a low-risk deployment repository.
During the next ring, queue age increases because the new pods take longer to pull. Capacity telemetry shows the reserve pool is shrinking, so rollout pauses. The operator pre-pulls the image on the relevant nodes and raises warm capacity within the approved limit. After a soak period with stable queues, the remaining production deployment group rotates. The old DR tag and ARC values are updated in the same change. Final evidence includes the deprecation responses, image digests, checksum verification, scan result, synthetic and representative workflow runs, queue/capacity graphs, group membership, and proof that a scale-from-zero test launches only the supported image.
Protect CI Capacity During Brownouts and Enforcement
GitHub's published timeline for GitHub Enterprise Cloud includes staged brownouts before full enforcement on September 25, 2026. The schedule lists registration brownouts on August 24, August 31, September 2, September 7, September 9, and September 11, with runtime blocking included on selected dates, followed by configuration plus runtime brownouts on September 14, 16, and 18. GitHub Enterprise Cloud with Data Residency had a separate full-enforcement date of July 31, 2026. GitHub Enterprise Server was explicitly outside this github.com enforcement change at the cited date.
Do not interpret a brownout as random flakiness. Model the four failure patterns: blocked registration, runtime rejection, backlog growth, and autoscaler churn. An autoscaler can make an incident worse if it repeatedly launches an image that cannot register, consuming compute quota without adding eligible runners. Stop the launch loop, switch the template to a supported image, and preserve enough capacity to drain the queue.
Failure mode | Capacity effect | Containment |
Registration blocked | New capacity never joins | Disable stale launch source; route to supported pool |
Runtime rejection | Connected capacity stops useful work | Drain/quarantine affected runners; replace |
Backlog growth | Developer delivery slows or times out | Prioritize critical workflows, cap nonessential concurrency |
Autoscaler churn | Compute cost/quota rises with no usable workers | Stop scale loop and correct image/template |
Reduced pool during upgrade | Less headroom for burst demand | Reserve supported warm capacity and stage rings |
Use a deadline-relative plan so the process remains useful before and after September 25, 2026. At T-minus several release cycles, inventory and eliminate unknown versions. Before the registration cutoff can affect recovery, rebuild every autoscaling and DR source. Before runtime enforcement, prove enough supported capacity to carry critical queues. At T-plus, any blocked or outdated runner is an incident/remediation item, not a migration exception. Rebuild from a supported source, deregister stale nodes, drain backlog, and verify that no recovery template can recreate the failing version.
· Stop rollout if supported spare capacity falls below the locally defined reserve or queue age accelerates unexpectedly.
· Reduce concurrency for noncritical workloads when doing so protects deployment, security, or release pipelines.
· Escalate if privileged or business-critical groups have no supported fallback capacity.
· Close incident only after new registration works, representative jobs run, backlog returns to normal, and stale launch sources are removed.
Prove Rollback Without Reintroducing an Unsupported Runner
Rollback is not 'deploy the previous artifact.' The immediately previous image may already be inside the registration or runtime danger window. A valid rollback target is a known-good image or installation set that is still supported, still compatible with the target scope, and still has enough deprecation headroom for the expected recovery period.
Rollback evidence | Why it matters |
Runner version and queried deprecation dates | Proves the fallback is still eligible to register and run |
Immutable image digest or VM image ID | Prevents alias drift during emergency use |
Runner package checksum / install media | Allows integrity verification if rebuild is required |
Dependency and certificate set | Avoids restoring a runner binary without its working environment |
Network/proxy policy revision | Preserves the connectivity state that was tested |
Controller/template revision | Restores ARC or autoscaling behavior, not only the binary |
Capacity reservation | Ensures fallback can absorb jobs |
Validation workflow result | Shows registration, labels, groups, and representative job passed |
For high-risk rings, test rollback before promotion. In a lab or non-production scope, deploy the candidate, execute the validation workflow, switch back to the known-good supported image, and execute the same validation again. If either direction requires undocumented manual repair, the rollback procedure is not production-ready.
Preserve a Known-Good Supported Image
Retain at least one verified fallback for each significant runner class until the new version has completed its soak and the fallback still remains inside the support policy. Keep the image digest, runner package, checksum, configuration, dependency versions, certificates, network policy, and automation revision together. A binary without its configuration is not a recovery artifact.
· Re-query the fallback version before a risky rollout or emergency rollback. Support dates can matter more than recency.
· Prevent cleanup jobs from deleting the only supported fallback during the soak window.
· Keep enough cloud quota, cluster capacity, or reserved hosts to instantiate the fallback.
· Quarterly, run a recovery exercise that scales the fallback, registers it, executes a representative job, and returns to the primary image.
· If the fallback approaches its own deadline, replace the rollback baseline proactively rather than discovering the gap during an incident.
GitHub Enterprise Server requires a separate governance branch. The github.com September 2026 enforcement does not automatically apply to GHES. Instead, align runner versions to the GHES release line and its documented minimum runner version. The GitHub Enterprise Server releases documentation publishes those compatibility minimums by GHES version. Keep GHES and GitHub Enterprise Cloud risk registers separate so a policy update for one service is not transferred to the other without evidence.
Operate Runner Lifecycle as a Continuous DevOps Control
The September enforcement should be treated as a trigger to establish a permanent control, not as a one-time cleanup. New releases will continue, image factories will change, teams will add specialist pools, clusters will cache old containers, and disaster-recovery automation will age. The control must continuously discover, evaluate, rebuild, roll out, and verify.
Cadence / trigger | Control action | Evidence retained |
Scheduled discovery | List runners, groups, labels, versions, status, busy/ephemeral state | Fleet snapshot and unknown records |
Runner release event | Check target-scope availability; open rebuild/update work | Release timestamp and expected version |
Daily or policy check | Query deprecation schedule for distinct versions | Raw API responses and risk register |
Image publish | Checksum, scan, smoke registration/job test | Digest, build record, test run |
Ring promotion | Drain/replace, verify capacity and workload behavior | Change record, queue and capacity telemetry |
Exception approval | Document owner, reason, expiry, fallback | Risk acceptance with automatic expiry |
Incident | Stop stale launch path, restore supported capacity, drain queue | Timeline, recovered version/image, closure proof |
Quarterly recovery test | Exercise rollback and scale-from-zero path | Registration/job evidence and remediation findings |
The control should integrate with the same platform pipelines that create images and infrastructure. That is where DevOps platform engineering and agentic pipelines context is useful, provided automation never bypasses release authority. A bot can discover a new runner release, query deprecation dates, build an image, and open a change. Promotion to privileged or business-critical groups should still honor policy gates, business validation, security approval where required, and stop conditions.
A practical ownership model is RACI-like without turning every patch into committee work: platform engineering is accountable for the runner binary, images, templates, and ARC; security owns critical-update policy and exceptions; SRE/operations owns capacity and incident response; application teams validate representative workflows; GitHub enterprise administrators own scope permissions, groups, and enterprise configuration; engineering leadership accepts residual business risk when deadlines cannot be met. Skills described in the path to a first DevOps role can help explain the foundations behind this work, but fleet governance requires explicit production ownership beyond role titles.
1. Discover live runners and every launchable artifact on a fixed schedule.
2. Query version-specific registration and runtime dates for every distinct version.
3. Trigger rebuilds or managed updates from release events before the internal deadline is threatened.
4. Promote through risk rings while preserving supported spare capacity and measurable stop gates.
5. Expire exceptions automatically and escalate critical security releases outside the normal cadence.
6. Retain enough evidence to reproduce both the primary and rollback images.
7. Run recovery tests that prove a scale-from-zero path does not resurrect a stale runner.
The lifecycle acceptance standard is stronger than a green runner list. Every runner class needs an owner, an observed version, a known image or installation source, measured registration and runtime headroom, a supported replacement, a capacity plan, tested network and label behavior, an actionable alert, and a rollback target that remains supported. A runner completing a job proves only that the present execution succeeded. It does not prove that a replacement can register after a scale event or that the same worker will remain eligible after the support floor moves.
Teams that want to strengthen the underlying skills can use the Refonte Learning DevOps Engineer Program as one structured path for Linux and scripting, Git and GitHub, continuous integration and continuous deployment, Docker and Kubernetes, Terraform, cloud platforms, and monitoring and logging. Those foundations support this operating model without implying that the program specifically teaches GitHub self-hosted runner administration or ARC governance.
