Articles · Trial

A one-month AI agent trial: one number, not impressions

12 October 2026

At a dental clinic with three chairs, the patient neither turns up nor gives notice for 11.4% of appointments — 126 appointments a month on average. A vendor is offering a reminder bot: 30,000 tenge a month plus 50,000 tenge to set up. The clinic is invented — it is one of the sample reports we use to test our own audit agent — but the owner's question is real: take it or not.

The clinic's data seem to be on the vendor's side: with a reminder, 5% of patients fail to turn up; without one, 17%. But the report carries a caveat that holds for any firm: "this proves a link, not a cause". An appointment is marked as reminded only when the patient was actually reached, and those may simply be the more reliable patients. So the report advises taking the bot only for a one-month trial, judged on a single number.

A trial is four things written down before it starts: the number, the term, the stop condition and the person who watches. Without them, all that is left after a month is impressions, and the vendor's impressions are always good.

Step 1. One number, with today's value written down

At the clinic it is the no-show rate: 11.4%. Not "convenience", not "things got easier for the receptionists", and not five indicators at once. Out of five numbers, one will improve by itself within a month, and that is the one that gets shown at the wrap-up meeting.

The number comes from your own system, not from the vendor's dashboard, and it is calculated the same way before and after. At the clinic, the report first asks for the reminder field to distinguish "not called" from "no answer": while the two are mixed together, there is no telling what actually changed.

And the event has to happen often. 126 no-shows a month is enough to see a shift. An event that happens five times a month will show nothing in a month.

Step 2. The term is one full cycle of the work, not "two weeks"

Two weeks sounds convenient: short and not alarming. But the term is set not by the vendor but by how long one full cycle of the thing you are measuring takes.

At the dental clinic the cycle is a week. Mondays are worse than the other days: 28.6% of appointments get a reminder and 18.4% are no-shows. The trial has to take in several Mondays, and a month gives four.

At the fitness club (also invented) fees are collected once a month, the retry is on the fifteenth, and the membership is ended on the last day of the month. A two-week trial would not see a single full cycle. The report advises looking after two debit cycles.

At the training centre (invented, like the rest) a normal week brings 71 enquiries, and each of the four enrolment weeks brings 165. It is during enrolment that the receptionists fall behind. A trial in a quiet month says nothing about enrolment.

Two weeks is enough when the event happens dozens of times a day and one week looks like the next. In every other case, a trial shorter than the cycle measures chance.

Step 3. The stop condition is written down before the start

Before you begin, write one sentence: "If by such-and-such a date the number is not below such-and-such, we switch it off." You choose the threshold. The clinic has a benchmark: among patients who were reminded, 5% fail to turn up. If the bot reminds everyone and the no-show rate is still around 11%, the reminder was never the issue.

The contract has to allow you to stop. The clinic's report sets a condition before any money changes hands: the vendor confirms that the bot can connect to the clinic's booking system. Ask in advance whether the set-up fee comes back if you stop. If it does not, that is the price of the answer, and it is better known before the start.

The training centre is being offered an AI receptionist: 150,000 KZT to set up, 70,000 KZT a month and a 12-month contract. That comes to 990,000 KZT in the first year. A trial can be stopped and this contract cannot, so it is a purchase, whatever it is called. Asking for "a month with the right to leave" costs nothing. A vendor's refusal is an answer too.

Step 4. The number is watched by someone who does not mind how it comes out

Not the vendor: the vendor's summary will show the number of messages sent, not the number of no-shows. And not the employee whose job is being changed. In a small firm that leaves the owner or the accountant: once a week, from the same export, by the same formula.

The same person makes sure that only one thing changes during the trial. The clinic's report puts a free step ahead of the bot: call Monday's patients on Friday. Bring in that rule and the bot on the same day, and a month later nobody will be able to say which of the two worked.

Next to the number, that person notes how much of your own people's time the trial took up. The clinic's bot takes away about 26 hours of reminder calls a month and adds 15 minutes a day for going through the replies.

Sometimes the trial is run by a person, not a program

At the service centre (invented; the report is here) 41.6% of enquiries come in during the evening, at night or on a Sunday. Of the enquiries made during opening hours, 42.5% end in an order; of the rest, 20.5%. A vendor is offering a bot: 150,000 KZT plus 25,000 KZT a month.

The report suggests something else: a month of evening and Sunday cover, done from a phone, for 60,000 KZT. Then compare the share of enquiries that end in an order with today's 20.5%.

What is being tested is not the bot but the assumption underneath it: an evening customer will buy if someone answers in the evening. The report says plainly that evening customers may be a different kind of customer. A person on cover tests that more cheaply than a bot does, and the cover ends by itself after a month. As for the bot, the vendor has not said whether it can be cancelled.

If a live person in the evening did not add orders, do not expect a bot to.

And if the person did, you now have a number to set against the price of the bot.

A trial has three outcomes, and "let's extend it" is not one of them

Keep, narrow, stop. Narrowing means leaving the agent the part of the work where the number moved and taking away the rest.

There is a fourth, honest one: not enough data. Then the trial is extended once, with a note of exactly what you are waiting for and until what date. "We'll extend it while things are unclear" turns a trial into a subscription.

The clinic's report sets the day in advance: come back to the decision a month after the bot goes live. It writes down the next fork in advance as well. If a third of calls are still being lost at peak times, compare a second receptionist for the peak hours at 90,000 tenge a month with the voice assistant at 60,000 tenge a month plus 200,000 tenge to set up. The condition is named before the number is known.

What to do today, before anything is bought

1. Work out your number for the past three months. Open your booking system or CRM. For a clinic it is appointments marked "no-show" divided by all appointments. Write the result down with the date.
2. Ask two members of staff how they set that mark. If they do it differently, agree on one way first. Otherwise the trial will measure the muddle.
3. Name your cycle. What repeats in your firm, and over what period: a week of appointments, a month of debits, an enrolment season. That is the shortest term for a trial.
4. Put two questions to the vendor by email. Whether you can leave after the first month. And what part of what you have paid will not come back if you do.

All articles