AI platform and identity teams reviewing OAuth consent, document permissions, and Bedrock knowledge-base retrieval tests in an enterprise workspace

User OAuth or Service Accounts? Choosing Identity for Bedrock Knowledge Bases

Mon, Sep 14, 2026

A knowledge-base prototype can appear healthy for months while resting on one employee's consent. The connector signs in, synchronization completes, retrieval returns plausible passages, and the demo becomes a departmental dependency. Then the employee changes roles, loses access, resets credentials, hits a conditional-access policy, or leaves the company. The refresh token stops working and nobody can explain who owns the connection. The more serious failure is quieter: the knowledge base retrieves a document that the requesting employee cannot open in SharePoint, OneDrive, or Confluence because source permissions were never preserved in the retrieval path.

That is why Bedrock Knowledge Base OAuth is not merely a setup choice. Connector identity, ingestion authorization, stored-index access, application authentication, query-time filtering, token lifecycle, synchronization, deletion, and evidence form one control system. AWS added user-managed three-legged OAuth (3LO) for SharePoint, OneDrive, and Confluence on September 4, 2026. It makes bounded experimentation faster, but it does not turn a human OAuth grant into a durable production identity or automatically preserve document permissions.

This guide treats identity as a production design decision. It gives teams a rubric for 3LO versus service identity, an inventory for consent and secrets, source-specific access-control tests, freshness and deletion service-level objectives (SLOs), retrieval leakage tests, promotion gates, incident procedures, and an exit plan. The central acceptance test is simple: every retrieved item must be authorized, current, attributable, and removable.

What User-Managed 3LO Changes in Knowledge-Base Setup

AWS's September 4, 2026 announcement of user-managed setup for SharePoint, OneDrive, and Confluence positions 3LO as the faster path for users who cannot or do not want to provision service credentials. In the console flow, the user signs in to the source, grants delegated access, and Amazon Bedrock Managed Knowledge Base stores refresh-token material in AWS Secrets Manager. The setup removes some up-front administrator work, but it transfers operational dependency to the consenting user's identity and repository access.

The right mental model is not 'Bedrock can now log in for me.' It is 'the knowledge base can act within the delegated user's current authority, until that authority or consent changes.' That distinction matters because the source account defines what can be ingested, while the knowledge-base application defines who can query the resulting index. Unless document-level access control is explicitly supported and enabled for the chosen connector and authentication mode, those two populations are not equivalent.

Keep retrieval state separate from conversational or agent state. Refonte Learning's discussion of AI agent memory systems is useful for understanding persistent state boundaries, but connector authorization is a different control plane: a passage can be remembered or indexed correctly and still be unauthorized for the person asking the question.

Control plane

Question to answer before production

Evidence

Source identity

Which user or service identity performs crawling and refresh?

Identity record, consent or app registration, source audit log

Ingestion scope

Exactly which sites, drives, spaces, folders, and content types can enter?

Connector configuration, source-side grants, test corpus

Index access

Who or what can call the knowledge base?

AWS IAM and application authorization policy

Query-time authorization

Are source document permissions filtered for the verified end user?

Connector ACL mode, identity mapping, negative tests

Lifecycle

What happens on revocation, offboarding, deletion, or tenant-policy change?

Runbook, alerts, recovery drill, deletion proof

Choose 3LO or a Service Identity With a Production Rubric

Do not choose an identity pattern by setup speed alone. Three-legged OAuth, an application identity such as Microsoft Entra app-only, and a source service account create different operational dependencies. The matrix below should be completed for each connector, not once for the whole knowledge base, because SharePoint, OneDrive, and Confluence do not expose identical authentication or access-control behavior.

Criterion

User-managed 3LO

App-only / service principal

Source service account

Setup speed

Fast; interactive sign-in

Slower; admin registration and permissions

Moderate; account plus credential issuance

Human dependency

High; tied to consenting user

Low; non-human identity

Medium; non-human account may still have human-managed credentials

Consent model

Delegated user consent; admin consent may be required by tenant policy

Administrator consent to application permissions

Source admin creates or authorizes account/token

MFA / conditional access

Can interrupt reauthorization or token use

Governed by workload/app controls rather than interactive user MFA

Depends on source policy and token type

Token lifecycle

Refresh token and reauthorization events

Certificate/client secret lifecycle

Password/API token/PAT lifecycle

Source scope

Bounded by user's access and connector filters

Bounded by app permissions plus site/drive grants

Bounded by service account permissions plus filters

Document ACL behavior

Do not assume; often unavailable in 3LO mode

Use when vendor documents ACL support

Connector-specific; may be required for ACL mode

Auditability

User identity is visible but owner changes are disruptive

Stable application identity and clear admin ownership

Stable name, but shared-account use can weaken attribution

Continuity

Weak unless ownership and reauthorization are engineered

Strong when credential rotation is operated

Strong if account is protected from human offboarding

Offboarding

Can break immediately or later

Independent of employee departure

Independent if account is truly non-human

Incident response

Revoke grant/token and quarantine index

Disable app, rotate certificate/secret, restrict grants

Disable account/token, rotate credentials, resync and retest

Use 3LO for Bounded, Reversible Prototypes

3LO is appropriate when the purpose is discovery: validate connector reachability, document formats, chunk quality, retrieval relevance, and operational effort against a small, non-sensitive corpus. The prototype must be explicitly reversible. Use a named test user, a small set of approved source locations, a dedicated non-production knowledge base, and a secret-name prefix that can be identified and deleted. Do not let the prototype token become the production credential by inertia.

  •         Time-box the authorization and record a planned teardown date.

  •         Exclude regulated, privileged, personnel, customer, legal-hold, and executive content unless the pilot's authorization semantics can be proven.

  •         Treat successful sync as a connectivity result, not a production security approval.

  •         Require a negative retrieval test showing that content outside the approved corpus is not retrievable.

  •         Define the migration target before the first pilot, especially when document-level permissions are a production requirement.

Use App-Only or Service Accounts for Durable Operations

A durable service identity is usually the better production choice when the knowledge base is expected to survive employee movement, run continuously, or enforce source-level permissions. For SharePoint, AWS documents Microsoft Entra ID App-Only authentication as the recommended method and the only SharePoint authentication method that supports document-level ACL behavior in the managed knowledge-base connector. The application uses certificate-based client credentials rather than an interactive user sign-in.

Service identity is not automatically least privilege. An app with tenant-wide read permissions can be more dangerous than a delegated user. Prefer per-site or otherwise bounded grants when the source supports them, assign an accountable owner and backup owner, set credential-expiry monitoring, and document who can change source permissions. Production readiness comes from durable identity plus bounded scope, not from replacing the word 'user' with 'service.'

Map Users, Tenants, Consent, and Administrator Prerequisites

Build an identity inventory before creating production connectors. The inventory is the control record that lets security, source owners, and platform teams answer who authorized the connector, where its authority comes from, and how it will be replaced. This is narrower than a general agent registry, although the ownership discipline aligns with AI agent sprawl governance.

Inventory field

Required record

Identity

User UPN/email, Entra application ID, or source service-account name

Tenant / organization

Microsoft tenant ID or Atlassian organization/site identifier

Source

SharePoint, OneDrive, or Confluence instance

OAuth / API scopes

Exact delegated or application permissions approved for this connector

Content scope

Sites, drives, spaces, folders, paths, labels, MIME types, date filters

Secret ARN

AWS Secrets Manager ARN storing refresh-token or connector credential material

KMS key

AWS managed key or customer-managed key and key owner

IAM roles

Sign-in caller role, knowledge-base execution role, incident-response role

Owner / backup owner

Named accountable roles, not only a project team

Consent date

Date and approver for user or administrator consent

Review date

Next access, scope, and owner review

Reauthorization

Runbook link or named procedure, with escalation path

Administrator prerequisites differ by source and tenant policy. Microsoft 365 may allow user consent or may require an administrator to pre-consent to the Bedrock application. Atlassian authorization controls can require a site administrator to approve the integration. A production record should capture both the technical consent and the policy decision that allowed it. Do not copy a SharePoint permission list into OneDrive or Confluence documentation; use the exact connector setup page for the authentication mode you actually deploy.

A practical acceptance gate is a consent review signed by the source owner and identity team. It should state the identity, tenant, requested scopes, content boundaries, document-level ACL mode, expected users, offboarding behavior, and emergency revocation method. If any field is unknown, the connector is still a prototype.

Own the Refresh-Token and Secret Lifecycle

With user-managed 3LO, the refresh token becomes a production credential even if Amazon Bedrock creates the Secrets Manager secret automatically. AWS documents that the generated secret lives in your AWS account and that the knowledge-base execution role needs access to it so tokens can be refreshed. That secret therefore belongs in your credential inventory, monitoring, incident response, and deletion process. Automatic creation is not automatic ownership.

The token can stop working for reasons outside the Bedrock configuration: user or administrator revocation, account disablement, source-side policy changes, tenant consent changes, security response, or provider-specific refresh-token rules. A resilient design assumes reauthorization will eventually be required. Track the last successful refresh indirectly through sync health and authentication errors, and set an alert before the business notices that the knowledge base has stopped changing.

Control Bedrock-Created Secrets, IAM, KMS, and Monitoring

Use a predictable secret-name prefix for each environment and connector where the 3LO flow supports it. Scope the setup operator and knowledge-base role to only the required secret paths. AWS's IAM guidance for user-managed 3LO setup separates permissions for the person initiating sign-in from permissions needed by the knowledge-base execution role. Preserve that separation so ordinary application operators cannot create new third-party grants or read unrelated credentials.

  •         Secret naming: include environment, source type, owning service, and a non-sensitive identifier.

  •         Encryption: document whether the secret uses the default Secrets Manager key or a customer-managed AWS Key Management Service (KMS) key; restrict key administrators separately from secret readers.

  •         IAM: allow only the specific Secrets Manager actions and resource patterns required for setup, refresh, and incident handling.

  •         Logging: retain AWS CloudTrail records for secret creation, update, access-policy changes, and deletion actions.

  •         Recovery: document whether reauthorization creates a new secret or updates an existing one, then test the exact process in non-production.

  •        Deletion: after connector retirement, delete the data source, revoke source authorization, delete unused secrets, and record evidence that the old credential can no longer be used.

The security mindset is similar to the one used for CI/CD secret leakage from AI coding agents, but the containment path is connector-specific: a leaked refresh token can expose source content, while a leaked application certificate or service-account token can expose the full scope assigned to the non-human identity.

Plan for Revocation, Password Changes, and Owner Departure

Create a lifecycle state model and assign an action to each transition. A user changing a password does not always produce the same result as an administrator revoking sessions, disabling the account, or changing conditional-access policy. Treat the source provider's actual token behavior as authoritative, and use authentication failures as a trigger for controlled reauthorization rather than repeated blind retries.

Event

Immediate action

Recovery evidence

Owner changes role

Review source permissions and business ownership before next sync

Owner approval plus scope diff

Owner leaves

Disable or quarantine 3LO connector; migrate to approved owner/service identity

Successful reauthorization, full sync, negative ACL tests

Consent revoked

Stop retrieval if freshness can no longer be trusted

New consent record and successful canary sync

Token/secret suspected leaked

Revoke at source, restrict secret IAM, quarantine index, rotate or reauthorize

Source revocation log, new credential, leakage regression tests

Tenant policy changes

Pause promotion or sync until consent and access policy are reconciled

Identity-team sign-off and successful policy-compatible login

Limit Sites, Drives, Spaces, Folders, and Content Classes

Least privilege begins before the first crawl. A connector identity can be narrow while the configured source scope is broad, or vice versa. Record both. For SharePoint, enumerate exact site URLs and item-path or date filters supported by the connector. For OneDrive, define which users' personal drives are in scope and use supported email, path, MIME-type, and date filters. For Confluence, enumerate the site, spaces, entity types, URLs, attachments, archived content, and MIME-type filters that are actually configured.

Do not rely on retrieval prompts to exclude sensitive classes. Exclusion belongs as close to the source as the connector allows. If a source cannot express a required filter, either segregate the content into a dedicated site/space, use a different connector path, or do not ingest it. A semantic layer can improve meaning and consistency, as discussed in semantic layers for AI agents, but semantic organization does not replace source authorization or minimization.

Scope dimension

Production rule

Site / drive / space

Allow-list only; record business owner and purpose

Folder / item path

Include only approved subtrees; test sibling paths are excluded

Content type

Decide files, pages, blogs, attachments, archived content separately

Sensitivity

Exclude high-sensitivity classes unless ACL enforcement and app authorization are proven

Retention / legal hold

Do not use the knowledge base to bypass source retention or hold requirements

External sharing

Test guest/external content explicitly; do not assume source sharing semantics map cleanly

Unsupported filters

Record as a risk and compensate with source segregation or a different ingestion pattern

Treat Document-Level ACL Support as a Design Constraint

Document-level access control must be an architecture input, not a feature you try to add after indexing. AWS explicitly warns in its SharePoint document-level access-control documentation that ACL awareness is filtering, not end-user authentication. Your application still has to authenticate the user, establish a trustworthy identity, and pass the correct identity context to retrieval. If the application can spoof or substitute that identity, connector-side filtering cannot create a secure boundary.

For SharePoint, ACL-enabled managed connectors require the documented app-only mode. OneDrive's 3LO documentation likewise states that user-managed setup does not support document-level ACLs, while the Microsoft Entra application mode is the documented path for ACL-aware retrieval. For Confluence, user-managed 3LO does not support document-level ACLs; AWS documents Basic authentication with additional Atlassian organization-admin credentials for ACL mode. These differences are not implementation trivia. They decide whether a shared knowledge base can safely serve users with different source permissions.

Do Not Assume OAuth Preserves Source Permissions

OAuth answers 'may this client act under this grant?' It does not, by itself, answer 'may the current end user see this indexed chunk?' In 3LO mode, the source sees the consenting user's delegated identity during crawl. If Bedrock indexes everything that identity can read and the knowledge base is queryable by a broader population, source permissions have been flattened unless a supported ACL feature adds query-time filtering.

Use a simple design rule: if two intended knowledge-base users have different rights to any document in the same source scope, require a connector/authentication combination with documented ACL behavior, or split the corpus into separately authorized knowledge bases. Do not depend on prompt instructions such as 'only answer with documents the user is allowed to see.' The model cannot infer source authorization from content safely.

Test Identity Matching and Query-Time Authorization

Build ACL tests from real permission structures, not only from synthetic public files. Test both positive access and denial. Run them before launch and after connector, identity-provider, group, or source-permission changes. The expected result for a denied document is no retrieval of its content, title, snippet, or revealing metadata.

Test case

Source setup

Expected retrieval result

Failure meaning

Direct allow

User A explicitly allowed on private document

Document may be retrieved for A

Identity mapping or ACL crawl broken if omitted

Direct deny

User B has no access

No document content or metadata for B

Leakage or wrong identity context if returned

Group allow

User A receives access through group

Allowed after membership is recognized

Group resolution failure if omitted

Removed user

Access removed after prior successful retrieval

Denied within defined permission-change SLO

Stale authorization if still returned

Newly granted user

Access added after prior denial

Allowed within defined SLO

Stale permission data if omitted too long

Inherited permission

Child item inherits parent/space rules

Matches source behavior

Inheritance mapping defect

Unique item restriction

Item breaks inheritance

Matches item-specific restriction

Flattened ACL risk

External user

Guest has narrowly scoped access

Only approved items returned

External identity mapping or tenant-boundary defect

Handle SharePoint, OneDrive, and Confluence Differences

The three Bedrock 3LO connectors share an interactive setup pattern, but their source models and production alternatives differ. Treat each as a separate integration. The connector-specific pages, including OneDrive user-managed 3LO and Confluence user-managed 3LO, explicitly warn that user-managed setup does not support document-level ACLs. That is the key production boundary for shared enterprise assistants.

Connector context also differs from tool invocation. Refonte Learning's article on no-code AI agents and MCP discusses tool and integration patterns; a repository connector is narrower. Its primary job here is to ingest and retrieve governed knowledge, not to give the model a general-purpose action channel.

Source

Identity options relevant here

Content scope

ACL-capable production path

Secret / credential behavior

SharePoint Online

User-managed 3LO; Microsoft Entra ID App-Only; legacy OAuth 2.0 option

Multiple sites, files/pages, supported path/date filters

Microsoft Entra ID App-Only with ACL enabled

3LO secret created by Bedrock; app-only uses managed secret plus certificate stored as configured

OneDrive for Business

User-managed 3LO; Microsoft Entra App ID; OAuth 2.0 option

Users personal drives with email/path/MIME/date filters

Microsoft Entra App ID with ACL enabled

3LO secret created by Bedrock; app mode uses explicitly managed credentials

Confluence Cloud

User-managed 3LO; Basic; OAuth 2.0 app

Site, spaces, pages/blogs/attachments and supported filters

Basic authentication plus required admin fields for ACL mode

3LO secret created by Bedrock; Basic stores user/API-token and ACL admin material when enabled

Known limitations belong in the design record. Examples include OneDrive content-model limits such as unsupported notebook types, source-specific filter capabilities, and authentication modes that cannot be combined with ACLs. If a business requirement depends on an unsupported combination, choose another identity pattern or another ingestion architecture rather than documenting the limitation as an acceptable surprise.

Apply Data Minimization, Classification, and Retention

A retrieval index is a derived copy of source information, with its own exposure paths and deletion duties. Minimize before ingestion. For each approved source area, record the business purpose, data classification, expected user population, retention obligation, legal-hold status, and whether the content may be used in model prompts or generated responses. If the same site mixes public handbooks with investigations, payroll, legal advice, or customer records, do not crawl the entire site simply because the connector identity can read it.

  •         Prefer allow-listed source containers over broad tenant or site roots.

  •         Exclude obsolete, duplicated, draft, superseded, or out-of-policy content when the connector exposes a reliable filter.

  •         Record unsupported minimization controls and compensate with source restructuring or a separate ingestion path.

  •         Do not use index retention to extend the life of source records that should have been deleted.

  •         Tag test corpora so security and legal reviewers can distinguish synthetic validation content from production records.

Acceptance gate: the source owner must be able to name every content class that can enter the index and every intended audience that can query it. If either answer is 'whatever the account can access,' the design is too broad.

Engineer Sync Freshness, Deletion, and Failure Recovery

Synchronization creates a temporal security boundary. A document can be created, modified, permissioned, or deleted in the source before the index reflects the change. AWS documents incremental synchronization for these managed connectors and provides scheduled synchronization options, but production teams still need measurable objectives. 'We sync daily' is a schedule, not evidence that deletion or permission changes converge within an acceptable time.

Measure four lags independently: create-to-retrievable, update-to-current, delete-to-not-retrievable, and permission-change-to-correct-access. Use canary documents with deterministic text and known permissions. Record source event time, sync start/end, first correct retrieval result, and any errors. For ACL-enabled connectors that perform real-time checks, permission-change behavior may differ from content-update behavior, so test it separately.

Metric

Example SLO to define locally

Alert condition

Recovery action

Create freshness

Approved content retrievable within N hours

Canary absent after SLO

Inspect sync status, source access, filters, throttling; resync

Update freshness

Changed text replaces old passage within N hours

Both old and new text remain retrievable

Quarantine affected source if stale answer risk is material

Deletion

Deleted item no longer retrievable within N hours

Deleted-content canary still hits

Stop retrieval for affected corpus, resync, escalate if removal stalls

Permission change

Denied/allowed result matches source within N minutes/hours

Negative ACL test fails

Disable affected connector or app path until identity/ACL issue is corrected

Authentication health

Every scheduled sync authenticates successfully

Repeated token/auth failures

Reauthorize or rotate credential; validate before reopening retrieval

Failure recovery must be idempotent and observable. Repeated sync attempts should not become your only strategy. Define when to resync, when to rebuild a data source, when to quarantine a knowledge base, and when to fail closed at the application layer because freshness cannot be trusted. Deleted-content hits should be treated as a security or records-management incident, not only a search-quality defect.

Evaluate Retrieval, Citations, Omissions, and Leakage

A grounded answer with a citation can still be wrong in four ways: the cited source may no longer exist, the user may not be allowed to open it, the indexed passage may be stale, or the system may omit a more authoritative document. Retrieval evaluation therefore needs more than relevance metrics. Build a test set that combines answer correctness with source existence, permission state, recency, and expected no-answer behavior.

Evaluation dimension

Test question

Pass condition

Grounded correctness

Does the answer follow the cited source rather than model prior knowledge?

Claim is supported by current source text

Citation validity

Does the citation resolve to the intended item?

Source exists and identifier/location is attributable

Permission

Can the authenticated user open the source item?

Retrieval matches documented ACL or corpus boundary

Deletion

What happens after the source item is deleted?

No answer or alternate authorized source after deletion SLO

Sensitive leakage

Can prompts elicit excluded names, snippets, titles, or metadata?

No protected information returned

Omission

Does retrieval find the controlling policy rather than an obsolete draft?

Authoritative source ranks or is included

No-answer

What happens when no authorized current source supports a response?

System abstains or clearly states insufficient evidence

Format/language

Do supported PDFs, office files, multilingual pages, and attachments retrieve correctly?

Expected content is retrievable without permission leakage

Run RAG data leakage tests with adversarial prompts such as requests for confidential titles, summaries of 'documents I cannot access,' names from private project spaces, or content previously retrieved before permission was removed. Include prompt variations and indirect questions. The test passes only when unauthorized source material is absent from both answer text and citations.

Keep a retrieval evidence record for high-risk applications: test-case identity, query, expected authorization outcome, retrieved source identifiers, source timestamps, answer result, and disposition. That evidence is what lets a team prove that a release improved authorization rather than only improving semantic relevance.

Promote Connectors Across Development and Production

Treat connectors as environment-specific infrastructure. Development and production should have separate identities, source scopes, secrets, knowledge bases or indexes, test corpora, and approval records. A user OAuth token from a successful prototype must never be copied forward as the production credential. Promotion should recreate the integration under the approved production identity and then rerun security and retrieval acceptance tests.

This fits the broader production discipline described in building AI agents in production, but the promotion gate here is connector-specific: the same prompt and model can be safe in a sandbox and unsafe in production if the production source scope or identity is broader.

Promotion gate

Required evidence

Identity

Production identity and backup owner recorded; no personal prototype token

Consent / app approval

Production tenant/site approval captured and independently reviewable

Source scope

Allow-list reviewed by knowledge owner and security

Secrets

Production secret path, KMS choice, IAM policy, and monitoring verified

ACL

Positive and negative tests signed off for the exact production auth mode

Freshness/deletion

Canary evidence meets local SLOs

Retrieval

Leakage, omission, citation, and no-answer tests pass

Rollback

Known way to disable connector/retrieval and return to previous approved corpus

Use change control for any production modification that can broaden access: app permission changes, new sites or spaces, new user-drive populations, ACL enablement decisions, source filters, secret policies, or application identity mapping. A retrieval-quality improvement does not waive an authorization review.

Monitor Incidents, Consent Drift, and Source Changes

Monitoring must combine AWS, source-platform, and application signals. A green knowledge-base endpoint says nothing about whether the source token still refreshes, whether the owner still exists, whether item counts collapsed, or whether sensitive content became retrievable. Build dashboards and alerts around the control objectives rather than only service availability.

Signal

Why it matters

Response threshold

3LO/authentication failures

Token revoked, expired, blocked, or policy changed

Alert on repeated failures or first failure for critical source

Sync status and duration

Detect stalled or degraded ingestion

Alert on missed schedule, abnormal duration, or persistent partial failure

Source throttling/API errors

Can create stale corpus without full outage

Alert when error rate threatens freshness SLO

Item-count change

Detect accidental broadening, mass deletion, or filter drift

Investigate material deviation from baseline

Deleted-content hit

Direct evidence of stale index or removal failure

Immediate incident for sensitive corpus

Retrieval denial/zero-result shift

May indicate identity mapping or ACL breakage

Investigate sudden cohort-level changes

Secret or IAM policy change

Credential exposure or unauthorized access expansion

High-priority security alert

Owner/account status change

3LO continuity and accountability risk

Trigger ownership review before or at offboarding

Consent drift deserves its own review. An administrator may change tenant-wide user-consent settings, block third-party applications, rotate organization API keys, alter site access, or change group membership rules. Keep a quarterly or risk-based reconciliation between the connector inventory and the identity/source systems, with faster checks for high-sensitivity corpora.

Design Migration, Reauthorization, and Data Exit

Production design is incomplete until you can replace the identity and remove the data. The most common migration is from a 3LO prototype to a durable service identity. Create the new connector in parallel where possible, ingest the same bounded corpus, compare item counts and retrieval behavior, rerun ACL and freshness tests, switch application traffic, then revoke and delete the old 3LO grant and secret. Avoid an in-place credential swap when it would obscure which identity produced which evidence.

1.        Inventory the current connector: source scope, identity, secret ARN, last successful sync, ACL mode, and dependent applications.

2.        Provision the target identity with the smallest source permissions that meet the requirement.

3.        Create a parallel non-production or shadow data source and ingest a controlled validation corpus.

4.        Compare source coverage, metadata, permission behavior, citations, freshness, and deletion propagation.

5.        Run the full negative leakage suite using identities from different permission groups.

6.        Approve cutover, switch the application to the target knowledge base or data source, and observe for a defined period.

7.        Revoke the old source authorization, disable/delete the old connector, remove unused secrets/certificates/tokens, and preserve deletion evidence.

For source-to-source migration, map document identifiers, owners, permissions, and retention state before re-ingestion. For platform exit, export the source inventory and evaluation corpus, rebuild retrieval on the alternate platform, compare authorization and deletion behavior, and then delete Bedrock data sources and credentials according to your retention policy. 'We stopped querying it' is not proof of data exit.

Exit artifact

Minimum proof

Source authorization

Grant/app/service-account credential revoked or disabled

AWS credential

Unused Secrets Manager secret and certificate material removed

Knowledge-base content

Data source or knowledge base deleted/retired per approved procedure

Application path

No production route references retired knowledge base

Evidence

Final item counts, deletion test, change ticket, owner sign-off

Run a 60-Day Identity-Aware Knowledge-Base Pilot

A good pilot is not an eight-week relevance demo. It is a controlled attempt to falsify the production design. Use bounded, representative content; named owners; a small set of test users with deliberately different permissions; and a known migration target. The timeline below assumes the team can obtain identity and source-owner participation. If those owners are unavailable, that itself is a no-go signal for production.

Days

Focus

Exit evidence

1-10

Scope and identity design

Decision matrix, identity inventory, source allow-list, risk classification, planned production identity

11-20

Connector setup and secret controls

Consent/admin approvals, IAM/KMS review, secret inventory, first successful bounded sync

21-30

ACL and authorization tests

Signed positive/negative matrix including removed/new users and permission changes

31-40

Freshness and deletion

Measured create/update/delete/permission-change lag; alerts and quarantine procedure

41-50

Retrieval evaluation

Citation, omission, leakage, no-answer, language/format test results

51-55

Incident drill

Leaked token or unauthorized retrieval tabletop/technical drill with revocation and containment

56-60

Migration and go/no-go

3LO-to-service-identity rehearsal where required, data-exit proof, production decision record

Go to production only if all critical controls have an owner and evidence. The minimum gate is: identity and consent are durable enough for the service; source scope is explicit; the selected connector/authentication mode provides the authorization semantics the application needs; negative ACL tests pass; freshness and deletion SLOs are measured; token or credential failure produces an alert; unauthorized retrieval has a containment path; and the team has rehearsed reauthorization or migration.

A pilot should end with one of three decisions: approve the production identity pattern, approve only a restricted corpus with compensating controls, or reject the connector design and choose a different ingestion/authorization architecture. Do not label a successful demo as a production approval when document permissions cannot be proven.

The production threshold for a managed knowledge base is higher than successful ingestion. Every source item needs a justified ingestion identity, a known source-authorization path, a current synchronization state, retrieval evidence for the requesting user, and a removal path. Three-legged OAuth can shorten the path to a bounded prototype, but convenience must not become an implicit security model. When continuity or document-level permissions matter, use the durable authentication mode that the source connector actually documents, then test it under real positive and negative identities.

Bedrock knowledge base identity governance is therefore an operating discipline: inventory identity and consent, minimize source scope, treat refresh tokens and service credentials as production secrets, measure deletion and permission-change lag, validate citations and leakage, separate development from production, and rehearse reauthorization and exit. If any layer is unverifiable, fail closed or reduce the corpus until it is.

Teams building these controls also need transferable skills in AI-system design, data engineering, scaling, evaluation, ethics, and governance. The Refonte Learning AI Engineering Program covers those foundations over a three-month program with an indicated 12-14 hour weekly commitment. The live curriculum does not explicitly list Bedrock Managed Knowledge Bases, 3LO connectors, or source ACL operations, so use it for the underlying engineering and governance skills rather than as a product-specific Bedrock certification path.