An AI agent’s knowledge base: what to put in it and how to maintain it

In a business, the expression “knowledge base” often refers to a help centre or an internal wiki, consulted by people. For an AI agent, the meaning is similar but the use differs: the base is read by a program which, for each question, looks for the relevant passages and uses them to write its answer. In this approach, it is not used to train the model: it is supplied to the model at the moment of answering, and can therefore be corrected at any time.

It brings together two types of content: information, that is, what you offer, on what terms, where and when; and instructions, which say how the agent should behave, what it should refuse and when it should hand over.

The principle has a name: retrieval-augmented generation, or RAG. The term comes from a paper by Patrick Lewis and his co-authors, presented at the NeurIPS conference in 2020. The authors note that large language models store knowledge in their parameters, but struggle to access it precisely, to show where their answers come from and to update that knowledge. They propose combining this internal memory with an external memory, an index of documents (Wikipedia articles in their experiments) queried at the moment of answering; on their generation tasks, the answers obtained were more specific, more diverse and more factual than those of a baseline model without retrieval.

A knowledge base does not eliminate all errors, however. A research paper published in 2025 (Kalai et al.) argues that language models sometimes guess when they are uncertain instead of acknowledging it, because their training and evaluation reward attempting an answer: if the information is missing from the base, the risk of an invented answer remains. The base must therefore be supplemented by an explicit instruction: say “I don’t know” and offer to pass on the request. OWASP also notes that retrieving documents does not fully prevent prompt injection, and that contradictory sources can cause knowledge conflicts.

Start from the questions your customers actually ask: messages received, calls, quote requests. For each one, check that the base contains an approved answer. It usually includes: a presentation of the business and its activities, the services and what they include, the area covered, opening hours and closure periods, how appointments work, the public terms (lead times, guarantees, documents to provide), and frequently asked questions with their answers.

Add the behavioural instructions: topics not to deal with, wording to avoid, commitments the agent must never make (a firm price, a guaranteed lead time) and what to do when a request goes beyond its scope. Also specify what the agent must collect before passing on a request: name, contact method, subject.

One piece of information per passage, worded so that it stands on its own: “We work in Toulouse and the neighbouring municipalities” is better than a link to a map. Prefer explicit headings, complete sentences and documents with selectable text rather than scanned images. Avoid complex tables, information scattered across several places and, above all, duplicates: two versions of the same price or the same opening hours will end up contradicting each other.

Date any content tied to a period, such as summer opening hours or a temporary offer, and remove it when it expires. If your website is used as a source, check that it is itself up to date: the agent will repeat its errors.

No customer personal data, and no secrets. The GDPR requires personal data to be adequate, relevant and limited to what is necessary in relation to the purposes: this is the data minimisation principle. A named customer history has no place in the base of an agent that provides information to the public. Nor should it contain passwords, access keys, internal security procedures or confidential financial information.

OWASP ranks sensitive information disclosure among the major risks of language model applications, and points out that instructions forbidding the model from revealing certain data can be circumvented. The safest approach is therefore for that data never to be accessible to the agent. The ANSSI, France’s national cybersecurity agency, asks for the AI system to take access rights and the “need to know” principle into account in the answers it gives: a user must not be able to obtain, through the agent, information they are not entitled to see.

Finally, be wary of documents whose origin you do not control. OWASP describes the case of a CV containing hidden text, written in white on a white background, that gives instructions to the system responsible for reading it: every document must be checked before it enters the base.

Appoint someone responsible for the base and set a review schedule. Every change in the business, whether a new service, a new price or an exceptional closure, must trigger an update. Regularly review the questions the agent could not answer: they reveal the gaps in your base.

Before going live, check that the real frequently asked questions all have an approved answer, that no information appears twice, that temporary content is dated, that no customer personal data and no secrets are included, that the instruction for not answering and passing the request on is written down, and that the agent has been tested with out-of-scope questions.

In Alyas, the knowledge base is built from your website address, free text or PDF documents. The agent, on the phone as on your website chat, answers from this information; if it does not know, it says so and offers to pass on the request by collecting contact details. On the phone, it announces that it is an AI and that the call is recorded; call recordings, transcripts and summaries are hosted in the European Union. You can try it for 10 days, with no credit card required.