Mechanical Turk Closes September 30: Rebuild Human Labeling Before the Workforce Disappears

A model team can appear operationally healthy even when its human-work layer depends on a service with a fixed closure date. One training pipeline may call Amazon Mechanical Turk directly through the Requester API. Another may send images to an Amazon SageMaker Ground Truth labeling job, while a production moderation service starts Amazon Augmented AI (A2I) human loops. All three can rely on the same public workforce even though their APIs, owners, task states, outputs, and quality controls differ.

Amazon set September 30, 2026 as MTurk's permanent closure date. On that date, workers can no longer submit Human Intelligence Tasks (HITs), remaining unsubmitted HITs expire, and the MTurk worker type stops being available for new Ground Truth labeling jobs and A2I human-review workflows. Changing a workforce Amazon Resource Name (ARN) or vendor endpoint is not a migration by itself. The replacement must reproduce enough capacity, worker eligibility, instructions, compensation logic, privacy boundaries, review semantics, label statistics, integration states, and audit evidence to keep the business decision safe.

This playbook works backward from the human judgment that must continue, whether an organization is still preparing for September 30 or remediating a missed dependency afterward. It builds an executable dependency register, classifies work as retire, automate, redesign, or migrate, selects a workforce architecture, reconstructs the quality system, and tests statistically comparable samples in dual run. It also separates production cutover from financial and evidence closeout because approvals, bonuses, refunds, billing, and transaction-history access continue on different clocks after the worker marketplace stops accepting submissions.

Define the Closure Scope, Deadlines, and Residual Access Windows

The controlling source is Amazon Mechanical Turk's closure FAQ, verified September 3, 2026. It states that MTurk permanently closes on September 30, 2026; HIT submission stops that day; and every remaining unsubmitted HIT automatically expires. Requesters may approve or reject completed submissions through October 30, 2026 and may award bonuses through October 30. Transaction history remains available until January 28, 2027. In the deadline-relative controls below, T-0 means September 30, 2026.

The FAQ's wording that closure "also applies" to SageMaker Ground Truth and A2I can be misread. Its next sentence defines the scope: the MTurk worker type stops being available when creating labeling jobs or human-review workflows. It does not say that Ground Truth or A2I as complete services close on September 30. Separate AWS documentation verified September 3, 2026 says Ground Truth is no longer open to new customers; existing customers can continue using it, while AWS plans security and availability work rather than new features. A2I carries the same service status. That distinction matters for teams considering a new long-term dependency.

The program manager owns the master closure calendar, but each workflow owner must calculate a last-publish date from T-0 and measured completion behavior. Promotion is blocked if the owner cannot show how outstanding work drains before the service cutoff. At and after T-0, rollback cannot mean "send it back to MTurk"; it must route to a tested secondary pool, pause the queue, or activate a controlled manual process.

Inventory Every Human-Work Dependency Across Direct and SageMaker Paths

Cloud inventory alone will miss surveys, enrichment tools, research scripts, content-review portals, and applications that call MTurk from non-AWS infrastructure. Start with a dependency register that combines account discovery, code search, billing, data lineage, and interviews with research and operations teams. The broader platform context in Refonte Learning's guide to landing a first AI engineering role is useful background, but this migration requires a much narrower production inventory.

Separate Direct MTurk Requester Integrations From SageMaker Workforces

Search source code, deployment manifests, secrets, notebooks, and job schedulers for MTurk SDK clients, the production requester endpoint, CreateHIT, HITTypeId, HITLayoutId, QualificationTypeId, and assignment approval calls. The official CreateHIT API reference shows why configuration capture matters: reward, assignment duration, HIT lifetime, maximum assignments, auto-approval delay, qualifications, review policies, requester annotation, and idempotency token all affect behavior. Record them as data, not screenshots.

Use paginated discovery and preserve raw JSON before transforming it. A practical starting sequence is:

aws mturk list-hits \
  --endpoint-url "$MTURK_REQUESTER_ENDPOINT" \
  --output json > mturk-hits.json

AWS_REGIONS=$(aws ec2 describe-regions \
  --query 'Regions[].RegionName' \
  --output text)

for region in $AWS_REGIONS; do
  aws sagemaker list-labeling-jobs --region "$region" --output json
  aws sagemaker list-workforces --region "$region" --output json
  aws sagemaker list-workteams --region "$region" --output json
  aws sagemaker list-flow-definitions --region "$region" --output json
done

The commands are discovery inputs, not proof of completeness. Correlate them with CloudTrail, billing, S3 prefixes, application logs, repository search, and named owners. The AWS CLI list-HITs documentation also warns operationally through its pagination model: a one-page export can silently undercount the estate.

Separate Labeling Jobs, Human Review Loops, Surveys, and Custom Microtasks

A Ground Truth labeling job applies task-specific labeling and may consolidate multiple annotations. A2I starts human loops from built-in integrations or from application logic for custom task types. Direct MTurk surveys and microtasks may have external survey redirects, custom HTML, longitudinal exclusion qualifications, or bespoke payment and approval logic. Put each into a separate workflow row even when the displayed question looks similar.

Inventory active and historical A2I flow definitions, then list human loops for every flow definition ARN using the A2I list-human-loops command. Promotion is blocked until every downstream consumer has an owner, schema, and replacement path. A queue with no visible caller is not safe to retire; it is an unresolved dependency.

Decide Which Work to Retire, Automate, Redesign, or Migrate

Do not begin procurement by asking which vendor most resembles MTurk. Begin by deciding whether each human judgment belongs in the target operating model after the service cutoff. For every workflow, score human necessity, required expertise, sensitivity, latency, volume, consequence of error, automation potential, and acceptable rework. Refonte Learning's data science practitioner guide describes the broader model lifecycle; this decision gate isolates the human-work component within it.

The decision record must name one disposition, one accountable owner, one deadline, and one reversible fallback. “Hybrid” is not a disposition unless the routing rule is explicit. For example, automate obvious duplicates, send ordinary ambiguous records to trained contractors, and escalate regulated or high-impact items to domain experts.

Block migration when a team cannot explain who consumes the label or what decision the label changes. This prevents spending scarce migration effort faithfully recreating low-value work. Conversely, do not automate merely because elapsed time is short. A model that has never been evaluated on the workflow's difficult slices is a new production risk, not a continuity plan.

Choose a Replacement Workforce Architecture

The MTurk service page, verified September 3, 2026, described a global, on-demand, 24x7 workforce. Treat that reach and elasticity as a capability to replace, not a marketing phrase to copy into requirements. A smaller private or vendor pool may be better qualified and more controllable, yet still fail overnight, multilingual, or burst-volume demand. Measure the exact operating envelope.

Compare Vendor-Managed and Private Workforces

AWS documents three SageMaker workforce patterns: public MTurk, vendor-managed, and private workforces. Its vendor workforce guidance makes the commercial boundary clear: the Marketplace subscription governs price, schedule, and refund policy, and operational issues are handled with the vendor. Procurement must additionally negotiate measurable capacity, named support, subcontractor disclosure, data deletion, audit evidence, model-training restrictions, and exit exports.

A private workforce uses work teams and can authenticate through Amazon Cognito or an organization's OpenID Connect (OIDC) identity provider, as described in the private workforce setup documentation. Test joiner, mover, leaver, multifactor authentication, session expiry, disabled users, and emergency access. The CreateWorkforce API reference states that only one workforce can exist in each AWS Region per AWS account, so architecture owners must inspect existing use before creating or changing it.

Use Hybrid Domain Experts and Escalation Pools Where Needed

A two- or three-tier architecture often controls cost without pretending every label has equal risk. Tier 1 can verify automated pre-labels. Tier 2 handles ambiguity with trained annotators. Tier 3 adjudicates disagreements or high-impact records with subject-matter experts. Routing should use observable conditions such as low model confidence, conflicting worker answers, prohibited content, rare label class, or a high-risk downstream action.

The acceptance gate is a staffed capacity plan for every tier, including nights, weekends, surge, absence, and training time. A vendor promise or employee headcount is not sufficient. Run the actual task, at the actual instruction version, with the actual data-access path and measure completed accepted labels per hour.

Rebuild Task Templates, Instructions, Qualifications, and Worker Experience

A task is an operating product. Migrating only question text loses examples, edge cases, layout behavior, qualification thresholds, worker filters, time estimates, escalation, accessibility, and compensation assumptions. This is especially dangerous for nuanced language work; Refonte Learning's article on sentiment analysis for sarcasm and nuance illustrates why superficially simple labels require explicit contextual guidance.

Create a versioned task package for every workflow:

Do not convert MTurk approval history into a universal competence score. It reflected a worker's prior marketplace behavior, not necessarily the replacement task, domain, language, or interface. Map each qualification to a new observable requirement and retest it. Preserve legacy worker IDs and qualification decisions only under approved retention and privacy rules; never attempt to deanonymize workers.

Promotion is blocked if first-time workers cannot complete representative tasks without live coaching, or if the median observed task time materially exceeds the compensation model. Roll back by pinning the previous template version, stopping publication of the defective version, and re-queuing only records whose outputs are demonstrably affected.

Recreate the Quality System Instead of Copying Only the UI

A workforce migration can preserve the label buttons and still destroy statistical comparability. The quality system includes qualification tests, gold items with known answers, duplicate annotation, worker-level monitoring, consolidation, confidence capture, adjudication, expert escalation, sampling, and appeals. Refonte Learning's guide to LLM evaluation pipelines provides useful evaluation context; the migration-specific task is to keep the human measurement process stable enough to compare old and new outputs.

AWS's public workforce documentation makes an important semantic distinction: Ground Truth can use multiple workers and annotation consolidation, while A2I does not support annotation consolidation. Therefore, a single generic “three workers and majority vote” rule is not an accurate migration design.

Design Gold Questions, Consensus, Confidence, and Adjudication

Use gold items that represent easy, ambiguous, rare, multilingual, and high-impact cases; do not publish only obvious examples that workers can memorize. Keep a holdout gold set for monitoring. Duplicate a statistically planned subset, not every record, and assign duplicates so the same worker does not see both copies when memory could bias agreement.

AWS explains in its annotation consolidation documentation that Ground Truth combines two or more annotations into a probabilistic label for supported predefined tasks. It also warns that more workers can improve accuracy while increasing cost. If the replacement lacks equivalent consolidation, implement and validate an explicit algorithm or preserve individual annotations for downstream adjudication.

Define stop conditions before the pilot: a material fall in gold accuracy, class-specific drift beyond the agreed tolerance, a surge in adjudication, unexplained worker concentration, or a severe error in a high-impact class. Recovery requires corrected instructions or routing, retraining, fresh blinded samples, and approval by the model or data owner. Never “average through” a critical error class to make an overall score pass.

Protect Identity, Sensitive Data, Worker Privacy, and Access

Migration is a mandatory privacy reassessment, not permission to copy the old task unchanged. AWS states that public MTurk should not receive confidential information, personal information, or protected health information, and Ground Truth or A2I tasks using that workforce require a declaration that input is free of personally identifiable information. Direct MTurk also prohibits collecting personally identifiable information from workers and restricts attempts to derive identity from Worker IDs.

AWS's vendor documentation specifically says requesters are responsible for checking a vendor's security and compliance practices and says not to share protected health information with that workforce. A private workforce may offer stronger identity and access control, but it is not automatically compliant. Assess data-processing terms, cross-border access, subcontractors, endpoint security, monitoring, deletion, worker confidentiality, and the applicable employment or contractor model with qualified counsel.

Block promotion if a worker can access a project outside the assigned team, if deprovisioning does not terminate access promptly, or if the data owner cannot explain every field shown in the task. Rollback means disable the work team or vendor route, revoke sessions and credentials, quarantine outputs, preserve incident evidence, and reissue affected records only after containment.

Model Capacity, Turnaround, Pricing, Payments, and Contract Risk

Unit reward is not unit cost, and nominal headcount is not capacity. Model arrivals, concurrency, task duration, acceptance, abandonment, rejection, rework, review, and peak demand. Use observed pilot distributions, including the 95th percentile task time, rather than a single optimistic average.

accepted_labels_per_hour =
  active_workers tasks_per_worker_hour
  acceptance_rate (1 - abandonment_rate)
  first_pass_acceptance_rate

cost_per_accepted_label =
  (worker_pay + platform_or_vendor_fees + onboarding + review
   + rework + payment_admin + minimum_commitments)
  / accepted_labels

A worked planning example, not a vendor benchmark: if 20 active workers average six tasks per hour, 90% of offered tasks are accepted, 8% are abandoned, and 92% pass first review, effective output is about 91 accepted labels per hour. A queue receiving 80 items per hour would appear stable on average but has little surge margin. Require a reserve based on peak arrivals, absence, and slower difficult slices.

The cutover gate is not the cheapest quote. It is sufficient measured throughput at acceptable quality, an approved total-cost model, and a signed path for disputes and exit. Rollback capacity must be funded, not merely listed in a slide.

Replace APIs, Queues, Webhooks, Status Handling, and Downstream Contracts

Do not let every application learn a new vendor's status model. Put a workforce adapter behind an internal task contract and publish canonical events. This separates business state from provider state and makes the second replacement less disruptive. The reliability patterns in building AI agents in production—versioned artifacts, canaries, kill switches, and human escalation—apply directly to these review queues.

MTurk's UniqueRequestToken helps prevent duplicate HIT creation only within a 24-hour token window, according to the official CreateHIT reference. Your internal idempotency key must live longer and cover publish, callback ingestion, approval, payment, and replay. Never infer success from a timeout. Query provider state by the stored external ID before retrying.

A provider-neutral event should carry at least task_id, workflow_id, provider, provider_task_id, state, event_time, template_version, schema_version, attempt, correlation_id, data_classification, and a content hash. Preserve raw provider payloads in controlled storage, then transform into the stable downstream schema.

{
  "task_id": "label-8f31...",
  "workflow_id": "returns-intent-v4",
  "provider": "replacement-a",
  "state": "SUBMITTED",
  "template_version": "returns-intent-v4.2",
  "schema_version": "label-event-v3",
  "attempt": 1,
  "payload_sha256": "..."
}

Promotion is blocked by duplicate downstream decisions, missing terminal states, unbounded retries, or schema differences not accepted by consumers. Before T-0, rollback may route eligible new work to MTurk only when measured pickup, completion, and review buffers fit before closure. At and after T-0, rollback must use the secondary route or safely accumulate a bounded backlog.

Dual-Run Old and New Workforces—or Reconstruct a Comparable Baseline

A convenience sample of fifty easy items does not demonstrate label parity. Before T-0, run stratified old and new arms across label class, ambiguity, language, domain, difficulty, worker exposure, and model-impact risk. After T-0, use frozen MTurk inputs, assignments, and adjudicated outputs as a retrospective baseline. Keep taxonomy, instructions, and adjudication stable where possible, and record unavoidable interface differences as treatment changes.

The data-science owner approves the analysis plan before replacement labels arrive or archived MTurk labels are unblinded. For high-impact binary classifications, consider a paired design and test discordant outcomes; for multiple raters or missing annotations, select an agreement statistic suited to the data rather than automatically reporting Cohen's kappa. Agreement is diagnostic, not the only success measure.

Measure Label Drift, Agreement, Bias, Throughput, and Rework

Use a dashboard that combines quality, operations, economics, and downstream effect. Refonte Learning's article on a definition of done for AI features makes the same essential point: a single average score cannot substitute for risk-aware acceptance.

Exit the live dual run or retrospective baseline comparison only when the predeclared sample is complete, severe defects are closed, capacity survives a peak test, privacy and procurement approvals are signed, and downstream consumers accept the new schema. A failing result triggers remediation and a fresh blinded sample; it does not justify moving the tolerance after seeing the data.

Export Tasks, Assignments, Qualifications, Payments, and Decision Evidence

The September 30 service cutoff is not the evidence deadline. MTurk says it removes HIT and assignment data after 120 days and recommends local download for longer retention, while transaction history remains available until January 28, 2027. Export before each platform access window closes and at the earliest remaining opportunity. If a window has already passed, treat the missing evidence as a records incident rather than assuming a later listing will reconstruct it.

Export with pagination and preserve both raw and normalized forms. For every HIT, retrieve assignments in all relevant states before deletion. Enumerate qualification types and associated workers where the API and policy permit. For A2I, preserve flow definitions and the S3 output objects; AWS's A2I output documentation describes output that includes the flow definition, human answers, submission details, and input context.

Create a manifest containing object path, size, export time, source account, source command or console report, SHA-256 hash, data classification, retention rule, and owner. Validate by selecting records from task creation through downstream dataset consumption and reconstructing the lineage without consulting MTurk. If reconstruction fails, the export is not complete.

Prepare for Capacity, Quality, Vendor, Identity, and Integration Failures

Before T-0, rehearse failures while both the MTurk path and the replacement are available. At and after T-0, rehearse the replacement and reserve paths with synthetic records under incident-remediation controls. The incident lead should own a one-page decision tree that says when to stop publication, how to drain or quarantine work, who can authorize a fallback, and what evidence must be retained.

Practice the cutover with synthetic records that exercise every canonical state, including duplicate callback, timeout after publish, late submission, rejection reversal, dead-letter replay, and provider outage. Run a separate privacy tabletop using a real contact tree but synthetic data.

A rollback is valid only if it is executable within the queue's recovery objective. “Call the vendor” is an escalation, not a rollback. Pre-provision the secondary route, identity, template, and minimum trained capacity; otherwise the honest fallback is to pause service and communicate the business impact.

Cut Over, Close Open Work, Reconcile Accounts, and Decommission MTurk

Cutover has two boundaries. The production boundary moves new work to the replacement. The residual boundary closes assignments, payments, refunds, billing, exports, and credentials after the September 30, 2026 service cutoff. Do not combine them into one ticket that is marked complete when the endpoint changes.

Before T-0, rollback may still send eligible new work to MTurk only when the workflow's measured pickup, completion, and review buffer fits before closure. At and after T-0, that route no longer exists. Keep replacement canary controls, but redirect failure to the reserve workforce or a bounded backlog rather than an unavailable service.

Do not delete A2I flow definitions merely to prove cleanup. AWS explains in its A2I workflow deletion guidance that non-started loops are stopped and deleted, while worker-started tasks may continue until completion or expiry; S3 answers are not removed by deleting the flow. Inventory and retain outputs first, then decommission according to the application's actual lifecycle.

The final MTurk decommission certificate should list zero active publishers, zero unresolved credentials, every residual financial owner, evidence locations and hashes, retention dates, known exceptions, and the executive who accepted any remaining risk.

Build a Deadline-Relative Migration and Post-Closure Remediation Plan

T-0 is September 30, 2026, the permanent service closure. Before T-0, execute the T-minus windows in order. After T-0, begin with containment, then complete any missed inventory, quality, privacy, contract, integration, and export gates under incident management. The migration lead tracks unknown dependencies and blockers in one decision register; time may compress, but required proof does not.

After T-0, use this immediate remediation sequence:

1. Isolate active publishers at feature flags, credentials, queues, or schedulers; capture logs before changing state.

2. Snapshot provider IDs, assignment and payment state, raw outputs, and downstream decisions; quarantine uncertain records.

3. Activate only a tested private, vendor, expert, or manual fallback; otherwise cap the backlog and disclose the impact.

4. Reconcile pre-closure submissions, auto-approval timing, permitted bonuses, refunds, and final invoices.

5. Search code, CloudTrail, billing, queues, and schedules again; sign proof of zero active publishers and owned exceptions.

October 30 and January 28 are closeout windows, not production extensions. If an access window passes before export completes, record the evidence gap, preserve remaining local records, and involve legal and records owners.

Every continuing workflow needs an owner, tested capacity, comparable quality, approved privacy and contracts, monitored integrations, and an executable fallback. Before T-0, these are promotion gates; after T-0, they are incident exit criteria. An endpoint change without them only defers the outage.

Longer-term capability requires statistical reasoning, data-quality judgment, machine-learning evaluation, visualization, and applied project discipline. The Refonte Learning Data Science & AI Program lists a three-month format with 12-14 hours of weekly dedication for learners engaged in bachelor's or postgraduate studies. Its verified competencies include Python data science, statistical modelling, exploratory data analysis, visualization, machine learning, predictive modelling, model optimization, and industry projects. The page does not claim platform-specific training in MTurk, Ground Truth, A2I, workforce procurement, or annotator operations.