A knowledge-base prototype can appear healthy for months while resting on one employee's consent. The connector signs in, synchronization completes, retrieval returns plausible passages, and the demo becomes a departmental dependency. Then the employee changes roles, loses access, resets credentials, hits a conditional-access policy, or leaves the company. The refresh token stops working and nobody can explain who owns the connection. The more serious failure is quieter: the knowledge base retrieves a document that the requesting employee cannot open in SharePoint, OneDrive, or Confluence because source permissions were never preserved in the retrieval path.
That is why Bedrock Knowledge Base OAuth is not merely a setup choice. Connector identity, ingestion authorization, stored-index access, application authentication, query-time filtering, token lifecycle, synchronization, deletion, and evidence form one control system. AWS added user-managed three-legged OAuth (3LO) for SharePoint, OneDrive, and Confluence on September 4, 2026. It makes bounded experimentation faster, but it does not turn a human OAuth grant into a durable production identity or automatically preserve document permissions.
This guide treats identity as a production design decision. It gives teams a rubric for 3LO versus service identity, an inventory for consent and secrets, source-specific access-control tests, freshness and deletion service-level objectives (SLOs), retrieval leakage tests, promotion gates, incident procedures, and an exit plan. The central acceptance test is simple: every retrieved item must be authorized, current, attributable, and removable.
What User-Managed 3LO Changes in Knowledge-Base Setup
AWS's September 4, 2026 announcement of user-managed setup for SharePoint, OneDrive, and Confluence positions 3LO as the faster path for users who cannot or do not want to provision service credentials. In the console flow, the user signs in to the source, grants delegated access, and Amazon Bedrock Managed Knowledge Base stores refresh-token material in AWS Secrets Manager. The setup removes some up-front administrator work, but it transfers operational dependency to the consenting user's identity and repository access.
The right mental model is not 'Bedrock can now log in for me.' It is 'the knowledge base can act within the delegated user's current authority, until that authority or consent changes.' That distinction matters because the source account defines what can be ingested, while the knowledge-base application defines who can query the resulting index. Unless document-level access control is explicitly supported and enabled for the chosen connector and authentication mode, those two populations are not equivalent.
Keep retrieval state separate from conversational or agent state. Refonte Learning's discussion of AI agent memory systems is useful for understanding persistent state boundaries, but connector authorization is a different control plane: a passage can be remembered or indexed correctly and still be unauthorized for the person asking the question.
Control plane | Question to answer before production | Evidence |
Source identity | Which user or service identity performs crawling and refresh? | Identity record, consent or app registration, source audit log |
Ingestion scope | Exactly which sites, drives, spaces, folders, and content types can enter? | Connector configuration, source-side grants, test corpus |
Index access | Who or what can call the knowledge base? | AWS IAM and application authorization policy |
Query-time authorization | Are source document permissions filtered for the verified end user? | Connector ACL mode, identity mapping, negative tests |
Lifecycle | What happens on revocation, offboarding, deletion, or tenant-policy change? | Runbook, alerts, recovery drill, deletion proof |
Choose 3LO or a Service Identity With a Production Rubric
Do not choose an identity pattern by setup speed alone. Three-legged OAuth, an application identity such as Microsoft Entra app-only, and a source service account create different operational dependencies. The matrix below should be completed for each connector, not once for the whole knowledge base, because SharePoint, OneDrive, and Confluence do not expose identical authentication or access-control behavior.
Criterion | User-managed 3LO | App-only / service principal | Source service account |
Setup speed | Fast; interactive sign-in | Slower; admin registration and permissions | Moderate; account plus credential issuance |
Human dependency | High; tied to consenting user | Low; non-human identity | Medium; non-human account may still have human-managed credentials |
Consent model | Delegated user consent; admin consent may be required by tenant policy | Administrator consent to application permissions | Source admin creates or authorizes account/token |
MFA / conditional access | Can interrupt reauthorization or token use | Governed by workload/app controls rather than interactive user MFA | Depends on source policy and token type |
Token lifecycle | Refresh token and reauthorization events | Certificate/client secret lifecycle | Password/API token/PAT lifecycle |
Source scope | Bounded by user's access and connector filters | Bounded by app permissions plus site/drive grants | Bounded by service account permissions plus filters |
Document ACL behavior | Do not assume; often unavailable in 3LO mode | Use when vendor documents ACL support | Connector-specific; may be required for ACL mode |
Auditability | User identity is visible but owner changes are disruptive | Stable application identity and clear admin ownership | Stable name, but shared-account use can weaken attribution |
Continuity | Weak unless ownership and reauthorization are engineered | Strong when credential rotation is operated | Strong if account is protected from human offboarding |
Offboarding | Can break immediately or later | Independent of employee departure | Independent if account is truly non-human |
Incident response | Revoke grant/token and quarantine index | Disable app, rotate certificate/secret, restrict grants | Disable account/token, rotate credentials, resync and retest |
Use 3LO for Bounded, Reversible Prototypes
3LO is appropriate when the purpose is discovery: validate connector reachability, document formats, chunk quality, retrieval relevance, and operational effort against a small, non-sensitive corpus. The prototype must be explicitly reversible. Use a named test user, a small set of approved source locations, a dedicated non-production knowledge base, and a secret-name prefix that can be identified and deleted. Do not let the prototype token become the production credential by inertia.
Time-box the authorization and record a planned teardown date.
Exclude regulated, privileged, personnel, customer, legal-hold, and executive content unless the pilot's authorization semantics can be proven.
Treat successful sync as a connectivity result, not a production security approval.
Require a negative retrieval test showing that content outside the approved corpus is not retrievable.
Define the migration target before the first pilot, especially when document-level permissions are a production requirement.
Use App-Only or Service Accounts for Durable Operations
A durable service identity is usually the better production choice when the knowledge base is expected to survive employee movement, run continuously, or enforce source-level permissions. For SharePoint, AWS documents Microsoft Entra ID App-Only authentication as the recommended method and the only SharePoint authentication method that supports document-level ACL behavior in the managed knowledge-base connector. The application uses certificate-based client credentials rather than an interactive user sign-in.
Service identity is not automatically least privilege. An app with tenant-wide read permissions can be more dangerous than a delegated user. Prefer per-site or otherwise bounded grants when the source supports them, assign an accountable owner and backup owner, set credential-expiry monitoring, and document who can change source permissions. Production readiness comes from durable identity plus bounded scope, not from replacing the word 'user' with 'service.'
Map Users, Tenants, Consent, and Administrator Prerequisites
Build an identity inventory before creating production connectors. The inventory is the control record that lets security, source owners, and platform teams answer who authorized the connector, where its authority comes from, and how it will be replaced. This is narrower than a general agent registry, although the ownership discipline aligns with AI agent sprawl governance.
Inventory field | Required record |
Identity | User UPN/email, Entra application ID, or source service-account name |
Tenant / organization | Microsoft tenant ID or Atlassian organization/site identifier |
Source | SharePoint, OneDrive, or Confluence instance |
OAuth / API scopes | Exact delegated or application permissions approved for this connector |
Content scope | Sites, drives, spaces, folders, paths, labels, MIME types, date filters |
Secret ARN | AWS Secrets Manager ARN storing refresh-token or connector credential material |
KMS key | AWS managed key or customer-managed key and key owner |
IAM roles | Sign-in caller role, knowledge-base execution role, incident-response role |
Owner / backup owner | Named accountable roles, not only a project team |
Consent date | Date and approver for user or administrator consent |
Review date | Next access, scope, and owner review |
Reauthorization | Runbook link or named procedure, with escalation path |
Administrator prerequisites differ by source and tenant policy. Microsoft 365 may allow user consent or may require an administrator to pre-consent to the Bedrock application. Atlassian authorization controls can require a site administrator to approve the integration. A production record should capture both the technical consent and the policy decision that allowed it. Do not copy a SharePoint permission list into OneDrive or Confluence documentation; use the exact connector setup page for the authentication mode you actually deploy.
A practical acceptance gate is a consent review signed by the source owner and identity team. It should state the identity, tenant, requested scopes, content boundaries, document-level ACL mode, expected users, offboarding behavior, and emergency revocation method. If any field is unknown, the connector is still a prototype.
Own the Refresh-Token and Secret Lifecycle
With user-managed 3LO, the refresh token becomes a production credential even if Amazon Bedrock creates the Secrets Manager secret automatically. AWS documents that the generated secret lives in your AWS account and that the knowledge-base execution role needs access to it so tokens can be refreshed. That secret therefore belongs in your credential inventory, monitoring, incident response, and deletion process. Automatic creation is not automatic ownership.
The token can stop working for reasons outside the Bedrock configuration: user or administrator revocation, account disablement, source-side policy changes, tenant consent changes, security response, or provider-specific refresh-token rules. A resilient design assumes reauthorization will eventually be required. Track the last successful refresh indirectly through sync health and authentication errors, and set an alert before the business notices that the knowledge base has stopped changing.
Control Bedrock-Created Secrets, IAM, KMS, and Monitoring
Use a predictable secret-name prefix for each environment and connector where the 3LO flow supports it. Scope the setup operator and knowledge-base role to only the required secret paths. AWS's IAM guidance for user-managed 3LO setup separates permissions for the person initiating sign-in from permissions needed by the knowledge-base execution role. Preserve that separation so ordinary application operators cannot create new third-party grants or read unrelated credentials.
Secret naming: include environment, source type, owning service, and a non-sensitive identifier.
Encryption: document whether the secret uses the default Secrets Manager key or a customer-managed AWS Key Management Service (KMS) key; restrict key administrators separately from secret readers.
IAM: allow only the specific Secrets Manager actions and resource patterns required for setup, refresh, and incident handling.
Logging: retain AWS CloudTrail records for secret creation, update, access-policy changes, and deletion actions.
Recovery: document whether reauthorization creates a new secret or updates an existing one, then test the exact process in non-production.
Deletion: after connector retirement, delete the data source, revoke source authorization, delete unused secrets, and record evidence that the old credential can no longer be used.
The security mindset is similar to the one used for CI/CD secret leakage from AI coding agents, but the containment path is connector-specific: a leaked refresh token can expose source content, while a leaked application certificate or service-account token can expose the full scope assigned to the non-human identity.
Plan for Revocation, Password Changes, and Owner Departure
Create a lifecycle state model and assign an action to each transition. A user changing a password does not always produce the same result as an administrator revoking sessions, disabling the account, or changing conditional-access policy. Treat the source provider's actual token behavior as authoritative, and use authentication failures as a trigger for controlled reauthorization rather than repeated blind retries.
Event | Immediate action | Recovery evidence |
Owner changes role | Review source permissions and business ownership before next sync | Owner approval plus scope diff |
Owner leaves | Disable or quarantine 3LO connector; migrate to approved owner/service identity | Successful reauthorization, full sync, negative ACL tests |
Consent revoked | Stop retrieval if freshness can no longer be trusted | New consent record and successful canary sync |
Token/secret suspected leaked | Revoke at source, restrict secret IAM, quarantine index, rotate or reauthorize | Source revocation log, new credential, leakage regression tests |
Tenant policy changes | Pause promotion or sync until consent and access policy are reconciled | Identity-team sign-off and successful policy-compatible login |
Limit Sites, Drives, Spaces, Folders, and Content Classes
Least privilege begins before the first crawl. A connector identity can be narrow while the configured source scope is broad, or vice versa. Record both. For SharePoint, enumerate exact site URLs and item-path or date filters supported by the connector. For OneDrive, define which users' personal drives are in scope and use supported email, path, MIME-type, and date filters. For Confluence, enumerate the site, spaces, entity types, URLs, attachments, archived content, and MIME-type filters that are actually configured.
Do not rely on retrieval prompts to exclude sensitive classes. Exclusion belongs as close to the source as the connector allows. If a source cannot express a required filter, either segregate the content into a dedicated site/space, use a different connector path, or do not ingest it. A semantic layer can improve meaning and consistency, as discussed in semantic layers for AI agents, but semantic organization does not replace source authorization or minimization.
Scope dimension | Production rule |
Site / drive / space | Allow-list only; record business owner and purpose |
Folder / item path | Include only approved subtrees; test sibling paths are excluded |
Content type | Decide files, pages, blogs, attachments, archived content separately |
Sensitivity | Exclude high-sensitivity classes unless ACL enforcement and app authorization are proven |
Retention / legal hold | Do not use the knowledge base to bypass source retention or hold requirements |
External sharing | Test guest/external content explicitly; do not assume source sharing semantics map cleanly |
Unsupported filters | Record as a risk and compensate with source segregation or a different ingestion pattern |
Treat Document-Level ACL Support as a Design Constraint
Document-level access control must be an architecture input, not a feature you try to add after indexing. AWS explicitly warns in its SharePoint document-level access-control documentation that ACL awareness is filtering, not end-user authentication. Your application still has to authenticate the user, establish a trustworthy identity, and pass the correct identity context to retrieval. If the application can spoof or substitute that identity, connector-side filtering cannot create a secure boundary.
For SharePoint, ACL-enabled managed connectors require the documented app-only mode. OneDrive's 3LO documentation likewise states that user-managed setup does not support document-level ACLs, while the Microsoft Entra application mode is the documented path for ACL-aware retrieval. For Confluence, user-managed 3LO does not support document-level ACLs; AWS documents Basic authentication with additional Atlassian organization-admin credentials for ACL mode. These differences are not implementation trivia. They decide whether a shared knowledge base can safely serve users with different source permissions.
Do Not Assume OAuth Preserves Source Permissions
OAuth answers 'may this client act under this grant?' It does not, by itself, answer 'may the current end user see this indexed chunk?' In 3LO mode, the source sees the consenting user's delegated identity during crawl. If Bedrock indexes everything that identity can read and the knowledge base is queryable by a broader population, source permissions have been flattened unless a supported ACL feature adds query-time filtering.
Use a simple design rule: if two intended knowledge-base users have different rights to any document in the same source scope, require a connector/authentication combination with documented ACL behavior, or split the corpus into separately authorized knowledge bases. Do not depend on prompt instructions such as 'only answer with documents the user is allowed to see.' The model cannot infer source authorization from content safely.
Test Identity Matching and Query-Time Authorization
Build ACL tests from real permission structures, not only from synthetic public files. Test both positive access and denial. Run them before launch and after connector, identity-provider, group, or source-permission changes. The expected result for a denied document is no retrieval of its content, title, snippet, or revealing metadata.
Test case | Source setup | Expected retrieval result | Failure meaning |
Direct allow | User A explicitly allowed on private document | Document may be retrieved for A | Identity mapping or ACL crawl broken if omitted |
Direct deny | User B has no access | No document content or metadata for B | Leakage or wrong identity context if returned |
Group allow | User A receives access through group | Allowed after membership is recognized | Group resolution failure if omitted |
Removed user | Access removed after prior successful retrieval | Denied within defined permission-change SLO | Stale authorization if still returned |
Newly granted user | Access added after prior denial | Allowed within defined SLO | Stale permission data if omitted too long |
Inherited permission | Child item inherits parent/space rules | Matches source behavior | Inheritance mapping defect |
Unique item restriction | Item breaks inheritance | Matches item-specific restriction | Flattened ACL risk |
External user | Guest has narrowly scoped access | Only approved items returned | External identity mapping or tenant-boundary defect |
Handle SharePoint, OneDrive, and Confluence Differences
The three Bedrock 3LO connectors share an interactive setup pattern, but their source models and production alternatives differ. Treat each as a separate integration. The connector-specific pages, including OneDrive user-managed 3LO and Confluence user-managed 3LO, explicitly warn that user-managed setup does not support document-level ACLs. That is the key production boundary for shared enterprise assistants.
Connector context also differs from tool invocation. Refonte Learning's article on no-code AI agents and MCP discusses tool and integration patterns; a repository connector is narrower. Its primary job here is to ingest and retrieve governed knowledge, not to give the model a general-purpose action channel.
Source | Identity options relevant here | Content scope | ACL-capable production path | Secret / credential behavior |
SharePoint Online | User-managed 3LO; Microsoft Entra ID App-Only; legacy OAuth 2.0 option | Multiple sites, files/pages, supported path/date filters | Microsoft Entra ID App-Only with ACL enabled | 3LO secret created by Bedrock; app-only uses managed secret plus certificate stored as configured |
OneDrive for Business | User-managed 3LO; Microsoft Entra App ID; OAuth 2.0 option | Users personal drives with email/path/MIME/date filters | Microsoft Entra App ID with ACL enabled | 3LO secret created by Bedrock; app mode uses explicitly managed credentials |
Confluence Cloud | User-managed 3LO; Basic; OAuth 2.0 app | Site, spaces, pages/blogs/attachments and supported filters | Basic authentication plus required admin fields for ACL mode | 3LO secret created by Bedrock; Basic stores user/API-token and ACL admin material when enabled |
Known limitations belong in the design record. Examples include OneDrive content-model limits such as unsupported notebook types, source-specific filter capabilities, and authentication modes that cannot be combined with ACLs. If a business requirement depends on an unsupported combination, choose another identity pattern or another ingestion architecture rather than documenting the limitation as an acceptable surprise.
Apply Data Minimization, Classification, and Retention
A retrieval index is a derived copy of source information, with its own exposure paths and deletion duties. Minimize before ingestion. For each approved source area, record the business purpose, data classification, expected user population, retention obligation, legal-hold status, and whether the content may be used in model prompts or generated responses. If the same site mixes public handbooks with investigations, payroll, legal advice, or customer records, do not crawl the entire site simply because the connector identity can read it.
Prefer allow-listed source containers over broad tenant or site roots.
Exclude obsolete, duplicated, draft, superseded, or out-of-policy content when the connector exposes a reliable filter.
Record unsupported minimization controls and compensate with source restructuring or a separate ingestion path.
Do not use index retention to extend the life of source records that should have been deleted.
Tag test corpora so security and legal reviewers can distinguish synthetic validation content from production records.
Acceptance gate: the source owner must be able to name every content class that can enter the index and every intended audience that can query it. If either answer is 'whatever the account can access,' the design is too broad.
Engineer Sync Freshness, Deletion, and Failure Recovery
Synchronization creates a temporal security boundary. A document can be created, modified, permissioned, or deleted in the source before the index reflects the change. AWS documents incremental synchronization for these managed connectors and provides scheduled synchronization options, but production teams still need measurable objectives. 'We sync daily' is a schedule, not evidence that deletion or permission changes converge within an acceptable time.
Measure four lags independently: create-to-retrievable, update-to-current, delete-to-not-retrievable, and permission-change-to-correct-access. Use canary documents with deterministic text and known permissions. Record source event time, sync start/end, first correct retrieval result, and any errors. For ACL-enabled connectors that perform real-time checks, permission-change behavior may differ from content-update behavior, so test it separately.
Metric | Example SLO to define locally | Alert condition | Recovery action |
Create freshness | Approved content retrievable within N hours | Canary absent after SLO | Inspect sync status, source access, filters, throttling; resync |
Update freshness | Changed text replaces old passage within N hours | Both old and new text remain retrievable | Quarantine affected source if stale answer risk is material |
Deletion | Deleted item no longer retrievable within N hours | Deleted-content canary still hits | Stop retrieval for affected corpus, resync, escalate if removal stalls |
Permission change | Denied/allowed result matches source within N minutes/hours | Negative ACL test fails | Disable affected connector or app path until identity/ACL issue is corrected |
Authentication health | Every scheduled sync authenticates successfully | Repeated token/auth failures | Reauthorize or rotate credential; validate before reopening retrieval |
Failure recovery must be idempotent and observable. Repeated sync attempts should not become your only strategy. Define when to resync, when to rebuild a data source, when to quarantine a knowledge base, and when to fail closed at the application layer because freshness cannot be trusted. Deleted-content hits should be treated as a security or records-management incident, not only a search-quality defect.
Evaluate Retrieval, Citations, Omissions, and Leakage
A grounded answer with a citation can still be wrong in four ways: the cited source may no longer exist, the user may not be allowed to open it, the indexed passage may be stale, or the system may omit a more authoritative document. Retrieval evaluation therefore needs more than relevance metrics. Build a test set that combines answer correctness with source existence, permission state, recency, and expected no-answer behavior.
Evaluation dimension | Test question | Pass condition |
Grounded correctness | Does the answer follow the cited source rather than model prior knowledge? | Claim is supported by current source text |
Citation validity | Does the citation resolve to the intended item? | Source exists and identifier/location is attributable |
Permission | Can the authenticated user open the source item? | Retrieval matches documented ACL or corpus boundary |
Deletion | What happens after the source item is deleted? | No answer or alternate authorized source after deletion SLO |
Sensitive leakage | Can prompts elicit excluded names, snippets, titles, or metadata? | No protected information returned |
Omission | Does retrieval find the controlling policy rather than an obsolete draft? | Authoritative source ranks or is included |
No-answer | What happens when no authorized current source supports a response? | System abstains or clearly states insufficient evidence |
Format/language | Do supported PDFs, office files, multilingual pages, and attachments retrieve correctly? | Expected content is retrievable without permission leakage |
Run RAG data leakage tests with adversarial prompts such as requests for confidential titles, summaries of 'documents I cannot access,' names from private project spaces, or content previously retrieved before permission was removed. Include prompt variations and indirect questions. The test passes only when unauthorized source material is absent from both answer text and citations.
Keep a retrieval evidence record for high-risk applications: test-case identity, query, expected authorization outcome, retrieved source identifiers, source timestamps, answer result, and disposition. That evidence is what lets a team prove that a release improved authorization rather than only improving semantic relevance.
Promote Connectors Across Development and Production
Treat connectors as environment-specific infrastructure. Development and production should have separate identities, source scopes, secrets, knowledge bases or indexes, test corpora, and approval records. A user OAuth token from a successful prototype must never be copied forward as the production credential. Promotion should recreate the integration under the approved production identity and then rerun security and retrieval acceptance tests.
This fits the broader production discipline described in building AI agents in production, but the promotion gate here is connector-specific: the same prompt and model can be safe in a sandbox and unsafe in production if the production source scope or identity is broader.
Promotion gate | Required evidence |
Identity | Production identity and backup owner recorded; no personal prototype token |
Consent / app approval | Production tenant/site approval captured and independently reviewable |
Source scope | Allow-list reviewed by knowledge owner and security |
Secrets | Production secret path, KMS choice, IAM policy, and monitoring verified |
ACL | Positive and negative tests signed off for the exact production auth mode |
Freshness/deletion | Canary evidence meets local SLOs |
Retrieval | Leakage, omission, citation, and no-answer tests pass |
Rollback | Known way to disable connector/retrieval and return to previous approved corpus |
Use change control for any production modification that can broaden access: app permission changes, new sites or spaces, new user-drive populations, ACL enablement decisions, source filters, secret policies, or application identity mapping. A retrieval-quality improvement does not waive an authorization review.
Monitor Incidents, Consent Drift, and Source Changes
Monitoring must combine AWS, source-platform, and application signals. A green knowledge-base endpoint says nothing about whether the source token still refreshes, whether the owner still exists, whether item counts collapsed, or whether sensitive content became retrievable. Build dashboards and alerts around the control objectives rather than only service availability.
Signal | Why it matters | Response threshold |
3LO/authentication failures | Token revoked, expired, blocked, or policy changed | Alert on repeated failures or first failure for critical source |
Sync status and duration | Detect stalled or degraded ingestion | Alert on missed schedule, abnormal duration, or persistent partial failure |
Source throttling/API errors | Can create stale corpus without full outage | Alert when error rate threatens freshness SLO |
Item-count change | Detect accidental broadening, mass deletion, or filter drift | Investigate material deviation from baseline |
Deleted-content hit | Direct evidence of stale index or removal failure | Immediate incident for sensitive corpus |
Retrieval denial/zero-result shift | May indicate identity mapping or ACL breakage | Investigate sudden cohort-level changes |
Secret or IAM policy change | Credential exposure or unauthorized access expansion | High-priority security alert |
Owner/account status change | 3LO continuity and accountability risk | Trigger ownership review before or at offboarding |
Consent drift deserves its own review. An administrator may change tenant-wide user-consent settings, block third-party applications, rotate organization API keys, alter site access, or change group membership rules. Keep a quarterly or risk-based reconciliation between the connector inventory and the identity/source systems, with faster checks for high-sensitivity corpora.
Design Migration, Reauthorization, and Data Exit
Production design is incomplete until you can replace the identity and remove the data. The most common migration is from a 3LO prototype to a durable service identity. Create the new connector in parallel where possible, ingest the same bounded corpus, compare item counts and retrieval behavior, rerun ACL and freshness tests, switch application traffic, then revoke and delete the old 3LO grant and secret. Avoid an in-place credential swap when it would obscure which identity produced which evidence.
1. Inventory the current connector: source scope, identity, secret ARN, last successful sync, ACL mode, and dependent applications.
2. Provision the target identity with the smallest source permissions that meet the requirement.
3. Create a parallel non-production or shadow data source and ingest a controlled validation corpus.
4. Compare source coverage, metadata, permission behavior, citations, freshness, and deletion propagation.
5. Run the full negative leakage suite using identities from different permission groups.
6. Approve cutover, switch the application to the target knowledge base or data source, and observe for a defined period.
7. Revoke the old source authorization, disable/delete the old connector, remove unused secrets/certificates/tokens, and preserve deletion evidence.
For source-to-source migration, map document identifiers, owners, permissions, and retention state before re-ingestion. For platform exit, export the source inventory and evaluation corpus, rebuild retrieval on the alternate platform, compare authorization and deletion behavior, and then delete Bedrock data sources and credentials according to your retention policy. 'We stopped querying it' is not proof of data exit.
Exit artifact | Minimum proof |
Source authorization | Grant/app/service-account credential revoked or disabled |
AWS credential | Unused Secrets Manager secret and certificate material removed |
Knowledge-base content | Data source or knowledge base deleted/retired per approved procedure |
Application path | No production route references retired knowledge base |
Evidence | Final item counts, deletion test, change ticket, owner sign-off |
Run a 60-Day Identity-Aware Knowledge-Base Pilot
A good pilot is not an eight-week relevance demo. It is a controlled attempt to falsify the production design. Use bounded, representative content; named owners; a small set of test users with deliberately different permissions; and a known migration target. The timeline below assumes the team can obtain identity and source-owner participation. If those owners are unavailable, that itself is a no-go signal for production.
Days | Focus | Exit evidence |
1-10 | Scope and identity design | Decision matrix, identity inventory, source allow-list, risk classification, planned production identity |
11-20 | Connector setup and secret controls | Consent/admin approvals, IAM/KMS review, secret inventory, first successful bounded sync |
21-30 | ACL and authorization tests | Signed positive/negative matrix including removed/new users and permission changes |
31-40 | Freshness and deletion | Measured create/update/delete/permission-change lag; alerts and quarantine procedure |
41-50 | Retrieval evaluation | Citation, omission, leakage, no-answer, language/format test results |
51-55 | Incident drill | Leaked token or unauthorized retrieval tabletop/technical drill with revocation and containment |
56-60 | Migration and go/no-go | 3LO-to-service-identity rehearsal where required, data-exit proof, production decision record |
Go to production only if all critical controls have an owner and evidence. The minimum gate is: identity and consent are durable enough for the service; source scope is explicit; the selected connector/authentication mode provides the authorization semantics the application needs; negative ACL tests pass; freshness and deletion SLOs are measured; token or credential failure produces an alert; unauthorized retrieval has a containment path; and the team has rehearsed reauthorization or migration.
A pilot should end with one of three decisions: approve the production identity pattern, approve only a restricted corpus with compensating controls, or reject the connector design and choose a different ingestion/authorization architecture. Do not label a successful demo as a production approval when document permissions cannot be proven.
The production threshold for a managed knowledge base is higher than successful ingestion. Every source item needs a justified ingestion identity, a known source-authorization path, a current synchronization state, retrieval evidence for the requesting user, and a removal path. Three-legged OAuth can shorten the path to a bounded prototype, but convenience must not become an implicit security model. When continuity or document-level permissions matter, use the durable authentication mode that the source connector actually documents, then test it under real positive and negative identities.
Bedrock knowledge base identity governance is therefore an operating discipline: inventory identity and consent, minimize source scope, treat refresh tokens and service credentials as production secrets, measure deletion and permission-change lag, validate citations and leakage, separate development from production, and rehearse reauthorization and exit. If any layer is unverifiable, fail closed or reduce the corpus until it is.
Teams building these controls also need transferable skills in AI-system design, data engineering, scaling, evaluation, ethics, and governance. The Refonte Learning AI Engineering Program covers those foundations over a three-month program with an indicated 12-14 hour weekly commitment. The live curriculum does not explicitly list Bedrock Managed Knowledge Bases, 3LO connectors, or source ACL operations, so use it for the underlying engineering and governance skills rather than as a product-specific Bedrock certification path.
