How to read an AI vendor's demo

McKinsey's late-2025 survey found 88% of organisations using AI in at least one function and only 7% scaled it across the business. A large part of that gap is bought in demos. The demo shows the script working. The business runs on the calls that go off script. Here are the six questions to ask, and what a good answer sounds like.

1. What happens when the caller goes off script?

Ask the demo to take a call about something it was not built for. A good vendor shows you the fallback: the engine says what it cannot do, collects the details, and hands off to a person with the context. A bad vendor changes the subject.

2. Where is the handoff, and how fast?

Every good deployment has rules for when a person takes over: emergencies, complaints, anything with legal or clinical weight. Ask to see the rules and the time it takes. If the answer is that the AI handles everything, you are being sold a liability.

3. What is the baseline?

Any promise of results without a baseline is a guess. A good vendor asks for your call, lead and hours data before quoting an outcome, and proposes to measure against it. A vendor quoting a recovery percentage in the first meeting has not looked at your business.

4. Whose data is it, and where does it live?

Your calls, your leads, your records. Ask where they are stored, who can see them, whether you can export everything on the day you leave, and what the vendor trains on. The answer should be short and written down.

5. Who fixes it at 2am?

Engines break in ways demos do not show: a calendar integration changes, a phone provider drops, a knowledge base goes stale. Ask who is on the hook, how you reach them, and what the caller hears while it is broken.

6. Show me the failure modes

Harvard Business School's 776-person field experiment found an individual working with AI matched a two-person team without it. That result came from real work with real failure modes, not a demo. A vendor who can tell you where their engine fails, and what they do about it, has run it in production. One who cannot has run it in a demo.

What to do with the answers

Write them down and hold the vendor to them in the contract: handoff rules, data terms, support terms, and a measurement against your baseline. Then decide. That filter applies to every vendor, including us.