How to Teach Your AI Agent to Answer Right: a Guide to RAG

There's one simple truth about AI agents: the agent is only as good as the information you fed it. You can connect the smartest model in the world - but if it doesn't know your price, your cancellation policy or the difference between your two products, it will give generic answers, dodge, or worse - make things up. The mechanism that solves this is called RAG (a knowledge base), and this guide explains, in the language of a business owner rather than an engineer, how to build it right.

What RAG is, in two lines

RAG (Retrieval-Augmented Generation) means: before the agent answers, it retrieves the relevant information from your documents and grounds its answer on it. Instead of "what the model knows about the world", the agent answers based on "what your business knows about itself". That's the difference between a temp who read one brochure and the veteran employee who knows every detail.

The four pillars of a good knowledge base

1. Price list and terms. The exact numbers, what's included and what isn't, discounts and payment terms. This is customers' number-one question - and a mistake here costs real money.
2. Real FAQs. Not what you think they'll ask - what they actually asked in the last month. Open your WhatsApp and email and take the 20 recurring questions.
3. Policies and procedures. Cancellations, refunds, warranty, delivery times, opening hours, service area. Anything with a "what happens if...".
4. Product and service details. Descriptions, differences between options, who each is right for. This is what lets the agent not just answer, but recommend and sell.

How to phrase the information so the agent understands

• Write like you'd explain to a customer, not like a legal document. Short, clear sentences in the language the customer speaks.
• Question-answer beats a paragraph. Instead of "our cancellation policy is flexible", write: "You can cancel up to 24 hours before the appointment at no charge. Less than 24 hours - full charge".
• Avoid contradictions. If one document says "one-year warranty" and another says "two years", the agent gets confused. One source of truth per topic.
• Update numbers, not templates. Prices, dates and promotions change - keep them in one easy-to-update place.

The three mistakes that ruin a knowledge base

How to prevent hallucinations (made-up answers)

Business owners' biggest fear of AI is that it will "make things up". The good news: with a solid knowledge base and a few steering rules, this almost never happens. Three mechanisms:

1. The agent is grounded in your documents, not a guess. When the answer is in the knowledge base - it retrieves it. When it isn't - it shouldn't invent one.
2. An explicit "don't know" instruction. You define for the agent: if you don't have the information, don't guess - say "let me check that and get back to you" and log the question.
3. A human handoff on sensitive topics. Special pricing, complaints, edge cases - go to you with full context, instead of the agent risking a wrong answer.

If you want to understand the risks and safety in depth - read our AI security guide, and if you're weighing different models - our AI model comparison.

Updating: the advantage no human employee gives

Changed a price? Added a service? A holiday promotion? You update the knowledge base - and the agent applies the change immediately, in the next conversation, across all channels at once. No handover period, no "I forgot to tell the team", no rep still quoting the old price. This is a point where an AI system is simply better than human knowledge management.

How much information you need - and what's included in the plan

For most small businesses, an effective knowledge base is 5-10 focused documents: price list, FAQs, policies and service descriptions. That's exactly why the Solo plan (490 ILS/month) includes a basic knowledge base of up to 10 documents - enough for a typical business. Businesses with large catalogs, complex price lists or many procedures move to the Pro plan (990 ILS) with an extended, unlimited knowledge base. In both cases, setup - including building the initial knowledge base - is done by our team and included in the price.

And if you're at an earlier stage and want to understand how to implement AI in your business the right way - we have a practical implementation guide that starts from the basics.

Frequently Asked Questions

What is a knowledge base (RAG) and why does it matter?

RAG (Retrieval-Augmented Generation) is a mechanism that lets the AI agent read from your business's documents and information and answer based on them, instead of relying only on general knowledge. It's what turns a generic agent into one that knows your specific price list, procedures and products - and stops it from making up answers.

What is most important to put in the knowledge base?

Four critical things: an accurate price list and terms; FAQs with the answers you actually give; policies (cancellations, warranty, delivery times); and product or service details. These cover the vast majority of customer conversations.

How do you stop the agent from making up answers (hallucinating)?

Three rules: feed only accurate, up-to-date information; instruct the agent to say "let me check and get back to you" when it doesn't have an answer instead of guessing; and define a human handoff for sensitive topics. An agent grounded in a good knowledge base and steered correctly rarely makes things up.

How often should the knowledge base be updated?

On any material change: new prices, an added service, a promotion, changed hours or procedures. The update itself is instant - the moment you update the knowledge base, the agent applies the change in the very next conversation.

Is a knowledge base included in the entry plan?

Yes. The Solo plan (490 ILS/month) includes a basic knowledge base of up to 10 documents, and the Pro plan (990 ILS) an extended, unlimited one - for businesses with large price lists, catalogs or many procedures.