Chat check
Your support bot
Paper 1Twenty customer questions
Please write clearly in block capitals.
I declare that I only used your public chat.
Your support bot sits an exam with every customer. I mark it.
I put 20 real customer questions to the chat on your website, mark every answer against your own help centre and policies, and show you exactly where it goes wrong, and why. The first check is free.
| Question | Mark | Question | Mark |
|---|---|---|---|
| 1 | 11 | ||
| 2 | 12 | ||
| 3 | 13 | ||
| 4 | 14 | ||
| 5 | 15 | ||
| 6 | 16 | ||
| 7 | 17 | ||
| 8 | 18 | ||
| 9 | 19 | ||
| 10 | 20 | ||
| TOTAL | 14 | ||
Example marks✓ correct ✗ wrong ~ incomplete
Instructionsfor your bot
- Answer all 20 questions.
- Use only what's on your company's own website.
- Don't make anything up. The examiner will check.
Informationfor you
- I use your public chat, like any customer. No logins, no system access.
- I ask hypothetical, before-you-buy questions. I never pretend to have an order, give personal details or ask for a human.
- Every answer is quoted word for word and marked against your own pages.
- You get a written report within 3 working days, at no cost.
Section A – Why good bots give wrong answers
Answer all questions in this section.
A customer asks your bot: “If I change my mind, how long do I have to return something?”
Read Sources A, B and C, which come from the same shop's website. What should the bot have said? Example
Source A
Refund policy · shop policy page
You can return any item within 14 days of delivery for a full refund.
Source B
Exchanges FAQ · help centre
Changed your mind? You have 30 days to exchange your item for a different size or colour.
Source C
Returns article · help centre
To start a return, fill in our returns form and we'll arrange a collection.No deadline?
The bot's answer
You have 30 days to return any item for a full refund.
0No. 30 days is for exchanges (Source B). Your refund policy says 14.
Why won't you see this in your reports?
The chat looks like a success.
A confident wrong answer counts as a resolved conversation. The customer only finds out when their return is refused, and by then it's a complaint, a phone call or a one-star review.
You pay for it anyway.
Many AI support tools charge for every conversation the bot resolves on its own. A wrong answer the customer accepts still counts.
Some answers carry legal weight.
Refunds, cancellations and deadlines touch customers' rights. Those answers need to match your published policy exactly.
Section B – Papers and fees
Start with Paper 1. It's free, and there's no obligation.
| Paper | What happens | Time allowed | Fee |
|---|---|---|---|
| 1 | Chat check FreeI write 20 questions your customers really ask, put them to your public chat and mark every answer against your own pages. The report shows each question, the bot's exact words, what your pages say and which page caused the confusion. |
3 working days | FreeNo obligation |
| 2 | Content fix Fixed priceI find every place your pages disagree and agree one clear answer per topic with you. Then I rewrite the help articles and policy wording, write guidance for your AI agent, and re-run the same questions so you can see the before and after. |
5 working days from getting access | £400One-off |
| 3 | Monthly upkeep OptionalPolicies change and new questions turn up. Each month I run a fresh 30-question test, review the chats your bot handed over to your team, update the content when a policy changes and send you a short summary. |
Every month | £200A month, cancel any time |
If Paper 1 finds nothing worth fixing, the report says so, and that's the end of it.
Report on the examination
What you get back: every question, the bot's exact words, the mark, and why.
- Paper
- 1, Chat check
- Candidate
- Support bot, online furniture retailer
- Marked against
- The retailer's own published pages
| Question | Mark | Examiner's comment |
|---|---|---|
| How long do I have to return something? | Said 30 days. The refund policy says 14. Two pages disagree. | |
| Can I collect my order from the showroom? | Offered collection. There isn't any, and showroom stock is display only. | |
| Can I pay in instalments with Amex? | Said yes. The payment page says no. The provider's article says yes. | |
| My delivery is the day after tomorrow. Can I still add assembly? | Right idea, but missed the 3-day notice rule. | |
| Do you deliver on Sundays? | Correct. Matches the delivery page. |
General comments
Polite, quick and confident. Too confident, in places: when the retailer's pages disagree, it picks one and says it like a fact. Four of the six answers that need fixing trace back to pages that contradict each other. Fix the pages and the marks will follow.
Illustrative example, not a real client.
Read the full sample reportThe examiner
I'm Andrew. I run Answer Key on my own, so the person who reads your help centre is the person who marks the answers, writes the fixes and re-runs the test.
I started it because AI support agents are only as good as the pages they read. Most help centres were written for people skimming one article, not for a bot that reads every page at once and takes all of it literally.
Rules I follow
I only use your public chat, the same as any customer.
I ask hypothetical, before-you-buy questions. I never pretend to have an order, give personal details or ask for a human.
I keep it to 20 questions over 4 chats, so testing costs you a few dollars at most.
Every mark points to the page it's based on. It's your policy, not my opinion.
Nothing on your site changes until you've approved every word.
AI helps with the heavy reading. I write the questions, and I check every answer and every fix myself.
Section C – Your questions
There are no wrong answers in this section.
1Do you need access to our systems for the free check?
No. I use your public chat and public help pages, the same as any customer. I'd only need access later, if you want me to make the fixes.
2Which platforms do you work with?
Any customer-facing AI agent with a public chat, including those built on Gorgias, Intercom Fin and Zendesk. The fixes are mostly to your content, so they carry over if you change tools.
3What if our bot is already getting things right?
Then the report says so, and you've got the evidence. If it only finds small problems, I'll tell you how to fix them yourself.
4Do you rewrite our legal terms?
No. I make your help content consistent with the policy you choose. If a policy itself raises a legal question, I'll flag it for your own legal adviser rather than give legal advice.
Your turn.
Send me your website address. I'll reply within one working day to confirm, and your report follows within three.
Or email me directly
contact@answerkey.co.uk- A reply within one working day
- Your report within three
- No logins, no access, no charge
Examiner's comment
Most wrong answers start like this. Help centres grow one article at a time. A returns window gets updated on one page and not another. A payment option changes, but the old FAQ stays up.
A person skims the one page they need. A bot reads all of them, and when they disagree it picks one and says it like a fact. Usually the fix is the content, not the bot.