AI Knowledge Bases: Decide Which Document Applies

Imagine a course business uploading its new handbook in October, ready for a November intake. The new students will submit assignments through a portal. The October cohort still uses email. Both documents are approved, and both describe the same course. An assistant answering “Where should I send my assignment?” needs to know which cohort is asking.

This fictional example exposes a management decision inside an apparently technical problem. Keeping an AI knowledge base useful involves deciding when a document applies, who can approve that decision and how the answer changes when the relevant context is missing.

Give the document a scope

A file’s upload date tells you when it entered storage. Its modification date tells you when it changed. Neither date, by itself, establishes which students, customers or projects should follow its instructions.

For the fictional course provider, the November handbook can be approved and available for advance enquiries while the October handbook remains the right source for existing students. Calling one document “current” across the whole business would hide that distinction.

I would start with a short record alongside each operational document:

• Audience: the course cohort, product version or project covered.
• Effective period: when these instructions begin and, where known, end.
• Approval: whether the content is a draft, approved for its stated use or withdrawn.
• Owner: the person authorised to resolve questions about its meaning.
• Relationship: which earlier document it replaces, and for which audience.

This is a proposed operating record. Its purpose is to make a business rule explicit before anyone translates it into software settings.

Make that scope affect retrieval

There are technical mechanisms for narrowing which documents a system retrieves. Amazon Bedrock’s configuration documentation describes filtering on document metadata, including matching attribute values and combining conditions. Its requirements and supported operators depend on the configuration and underlying store.

In our example, an implementer could use a confirmed cohort identifier to restrict the eligible handbook. That requires the identifier to be recorded correctly and the application to apply the intended filter. A label attached to a file is not, on its own, evidence that the application enforces the rule.

Where a chosen tool cannot provide the necessary separation, consider restricting each workflow to a controlled set of documents. Test the actual arrangement. The operational requirement remains clear: an October student should receive the October submission route, even when a newer handbook is available elsewhere.

Check that changes reach the searchable copy

The source folder and the material available to answer questions can be at different stages of an update. AWS’s knowledge-base synchronisation documentation says additions, modifications and removals in a connected data source require a sync so that the knowledge base is re-indexed. Its incremental process handles changes since the previous sync.

The same documentation notes that, in some supported storage configurations, newly synced information can take a few minutes to become available for queries. A completed upload therefore should not be your final acceptance check.

For the course provider, the practical check is to ask an assignment-submission question after the update and inspect the retrieved source as well as the answer. Record which handbook was actually used. This gives the team evidence about the path from the approved document to the response.

Treat withdrawal as its own operation

Microsoft’s Azure AI Search guidance on changed and deleted storage content, updated on 2 June 2026, illustrates a different maintenance issue. For the Azure Storage indexers it describes, change detection uses storage timestamps, while deletion detection needs an explicit strategy. Without appropriate handling, search documents can remain after their source objects have been deleted.

This is a platform-specific example, rather than a description of every AI tool. It demonstrates why a team should verify the removal path separately from the upload path. Ask the implementer what retires material from retrieval, how completion is checked and what happens to already indexed content.

Withdrawal from routine answers also needs a business definition. The October handbook should remain eligible for October students while they need it. Once it serves only historical purposes, the team can preserve its record separately from the documents used for routine support.

Separate content authority from system operation

Someone must decide which instructions apply; someone must implement that decision. In a small business, one person may perform both jobs, but the two checks remain distinct. Successfully processing a document does not establish that its instructions are approved for the relevant audience.

The content owner can confirm the cohort and effective period. The system operator can confirm ingestion, filtering and withdrawal behaviour. Agree who can pause an affected answer route if those checks disagree. MaryChuks.com’s decision-rights guide provides a useful way to name the owner, contributors and escalation boundary.

Test the boundary between versions

A compact acceptance check for the fictional course service could cover four situations:

• An October student asks where to submit: the expected source is the October handbook.
• A November student asks the same question: the expected source is the November handbook.
• A person gives no cohort: the expected next step is a clarifying question.
• The available documents conflict for the same cohort: the expected next step is referral to the content owner.

Decide these expected outcomes before running the checks. Then compare the answer, source and any clarification with that record. A correct-looking response supported by the wrong handbook deserves investigation: it may conceal a routing problem that becomes visible on the next question.

Review when the business changes

Connect document review to real events: a new intake, a product revision, a changed support route or an owner leaving the team. Give each change a named reviewer and record when the revised sources are ready for use.

The useful management question is whether the team can explain why this document applies to this question. When that explanation is visible, maintaining the knowledge base becomes a defined operational responsibility, with evidence that the intended instructions reach the intended people.

Sources

• AWS: Configure and customise queries and response generation — checked 24 October 2026.
• AWS: Sync your data with your Amazon Bedrock knowledge base — checked 24 October 2026.
• Microsoft: Change and delete detection using indexers for Azure Storage in Azure AI Search — updated 2 June 2026.