AI with a KPI: decide what the AI will do, and when, before you pick a tool

The most common way to start an AI project in a small business is to pick a tool. The owner sees an impressive demo, hears that a competitor is already "on AI", and asks for a quote for "a smart WhatsApp bot" or "AI for the website". It looks perfectly sensible, because that is how you buy almost any other technology: a computer, accounting software, security cameras. But AI is not equipment that does one thing decided in advance. In a conversation it can do almost anything, so if nobody decided what it will do, it does a bit of everything and nothing all the way. This article goes the other way round: the action, the moment and the number first, and only then the tool. We will cover how to set a KPI for each action of an AI agent, which seven actions you can define today, and when AI simply will not pay for you.

Why "we want AI" is still not a brief

"We want AI" is not a request, it is a feeling. It says something in the business is not working well enough, but not what. The difference shows the moment you try to write it down as a task. "A bot that answers questions" defines nothing you can check: which questions, how fast, and what should happen after the answer? An agent that answered a hundred questions about opening hours beautifully and booked no appointment did some work, but did not bring in a shekel.

The distinction that helps here is between a capability and an action. A capability is something the system is able to do: understand Hebrew, read a voice message, find an answer in the business's own documents. An action is something it does in your business, at a specific moment, with a result you can count. Demos sell capabilities, because they look impressive on screen. A business should buy actions, because only actions show up in the till.

One more advantage: once the action is defined, you can see in advance whether it is worth anything. If most of your enquiries come by phone, an agent on WhatsApp will not move your main number, and it is better to find that out at the planning stage than in month four.

The formula: action, moment, number and owner

Every action of an AI agent can be written as one sentence with four parts: "When [moment], the agent [action], we count [number], and [owner] is responsible for what happens next."

• The moment is the trigger. A first message came in. A customer asked about price. A day before an appointment. A customer went silent for two days after receiving a quote. Without a defined moment, the action is a capability waiting for someone to remember it.
• The action is one verb: answers, asks, books, reminds, sends, hands over. Not "handles", not "manages", and not "improves the customer experience".
• The number is what you will count after a month. Not a feeling and not "customers are happy", but something that appears in a table: time to reply, a percentage, a count.
• The owner is the person in the business who receives what the agent prepared. A hot lead handed to nobody is a lost lead with a nice summary.

Here is an example written out in full: "When a first enquiry arrives outside working hours, the agent replies within seconds, asks about the service and the preferred time, and offers two free slots from the calendar. We count how many of those enquiries became a booked appointment. The office manager goes through the night's bookings in the morning and deals with exceptions." You can build, test and fix that sentence. You cannot do any of that with "AI for WhatsApp".

Seven actions you can define today, and the KPI for each

These are the actions a WhaleBiz AI agent performs in businesses today, each with its natural moment and the number worth counting. Not every business needs all of them, and most start with two or three.

Three notes, because the details are what separate a real KPI from decoration.

"Replies within seconds" is an action only if the reply moves something forward. Speed is a condition, not a result. We wrote about the 5-minute rule and what every minute of delay costs, but even a reply in the first second is not worth much if it ends with "we will get back to you". So the important number is not only the reply time, but how many of the quickly answered enquiries reached the next step.

"Qualifies" is measured by how complete the card is, not by the number of questions. An agent that collected the name, phone, the service wanted and the next step did the job, even if it asked only three questions. A salesperson who gets a card like that in the morning does not need to call and ask everything again.

"Hands over to a person" is a complete action, not a failure of the agent. Every business has moments when a person has to step in: a large customer, a complaint, a medical or legal question the agent does not answer. The KPI here is not how few handovers there were, but how quickly a person received the conversation with all its context, and how many of those handovers ended in a deal.

Before you start: the baseline number

A KPI without a starting point is just a number. If after a month you find that 60% of enquiries became appointments, you will not know whether that is an improvement or a drop, unless you counted first what it was without the agent. So the first step is two weeks of simple counting, by hand or in a spreadsheet:

• How many enquiries came in, and through which channel.
• How many of them arrived outside working hours, including Friday, Saturday and holidays.
• How long it took, on average, until someone replied.
• How many enquiries became an appointment, a meeting or a payment.

Here is a worked example, with numbers we chose for illustration only and that you replace with your own. A business gets 200 enquiries a month, and 35% of them arrive outside working hours, which is 70 enquiries. Say that today half of them are answered only the next day, and many of those people have already found someone else. If 30% of enquiries answered on time become an appointment, and an appointment is worth 500 ₪ to the business on average, then the 35 late-answered enquiries are worth up to about 5,250 ₪ a month. That is not a promise, it is a ceiling. But now you have a KPI that means something: the share of off-hours enquiries that got a reply within a minute and became an appointment, and the value of those appointments.

We covered measurement after launch, including return on investment and the indirect metrics, in how to measure the true ROI of AI solutions. Here the point is the order: choose the number before you build, so the build serves it and not the other way round.

Where AI does not pay, and it is better to say so up front

Not every business needs AI, and not everywhere. These are the cases where we tell a customer to wait, or not to start at all:

• Few enquiries, and someone already replies fast. If ten enquiries come in a month and the owner answers all of them within minutes, there is no problem here for AI to solve. The number is already good.
• One long sale of a large contract. When every deal is months of negotiation with several decision makers, the agent can help with the first enquiry and with scheduling meetings, but the main value sits with the person who manages the relationship. Do not expect the main KPI to move because of the agent.
• A decision that needs professional judgement. A diagnosis, medical or legal advice, a quote that depends on an on-site inspection. The agent does none of these, and rightly so: it collects the details and hands over to a person. If most of your enquiries are like that, the action you measure is the quality of the handover, not closing.
• A broken process. If there is no up-to-date calendar, prices are unclear even to the team, and nobody owns a lead once it arrives, AI will only make the mess happen faster. Fix the process first, then speed it up.

Where AI sells, it pays. Where it does not, it is better to save the money.

From KPI to plan: the actions choose the plan

Once the actions are written down, choosing a plan almost does itself, because our plans are built around actions and channels, not around an amount of "intelligence".

• If your KPI is a fast reply and qualification on the website or in Telegram, Start is enough. It is a website widget or Telegram, with no WhatsApp, no appointment booking and no payments.
• If one of the actions is booking an appointment, or your customers write on WhatsApp, the entry point is Standard: that is where WhatsApp, calendar booking and sending media begin.
• If you need to send a payment link inside the conversation, reply on Instagram and Facebook too, or run several agents and several calendars for several team members, that is Business.
• Whatever does not fit the three boxes, such as a connection to an internal system or a unique process, is an Enterprise project and custom development, priced after a discovery call.

Prices and quotas are updated from time to time, so they live only in the plan comparison on our site. A few rules do not change: the setup fee is paid in one payment before work starts and is non-refundable, the monthly subscription starts on the day the agent goes live and not on the day you pay, and setup time is counted from the moment we have received all your materials. The first month after go-live includes revision rounds, and that is exactly the time to compare the numbers to the baseline and tune the actions. One more detail that matters when you count: a message for us is one agent reply, even when it is a reply to a voice message or a photo, and messages the customer sends are not counted.

What WhaleBiz does today, and what is still vision

WhaleBiz is an AI startup that wrote and runs its own sales system: AI agents on WhatsApp, the website, Instagram, Telegram and Facebook, with a built-in WhaleBiz CRM. Today the agent performs the seven actions in the table, in Hebrew, Russian and English. What it does not do is defined in the same detail: it gives no professional advice, promises no result, and does not carry on alone when a person has to decide. The model is hybrid: the AI handles most conversations, and a person from your team steps in when needed.

And because the agent runs the conversation itself and every conversation is recorded in the CRM, you see the numbers in the table from inside the conversation, not only from the click or the form. How a sales agent performs these actions end to end is explained on the WhaleBiz sales agent page, and if you are starting with WhatsApp, the guide to sales automation with a WhatsApp bot shows what the actions look like inside a conversation.

What is still vision: an agent you configure by chatting with it, and a system that coordinates a whole team of agents by itself. Today these are not part of the product, so we do not build a KPI on them.

Frequently Asked Questions

What is a KPI for an AI agent?

One number that shows whether a specific action of the agent happened and brought a result. For example, the share of enquiries that became an appointment, the time to first reply, or the share of payment links that were paid. A good KPI is tied to a defined action and moment, and is measured against a baseline you counted before launch.

How many actions should I define at the start?

Two or three. Most businesses start with a fast reply, qualification, and booking an appointment or handing over to a person. Once the numbers for those actions are stable, add the next action, such as reminders or sending media. An action added without a baseline is hard to measure.

How soon will I see whether the KPI moves?

The fast actions, such as reply time and the share of off-hours enquiries that got an answer, show in the first weeks. Numbers that depend on the sales cycle, such as appointments that actually took place or deals that closed, need at least one full sales cycle. The first month after go-live is the time to compare to the baseline and tune.

Are there businesses that should not bring in AI?

Yes. A business with few enquiries that the owner already answers within minutes, a sale that is entirely a long negotiation with one customer, or a process that is not yet in order, such as an outdated calendar and unclear prices. In those cases it is better to fix the process first, or not to invest at all.